-- Logs begin at Wed 2022-08-24 05:04:47 UTC, end at Wed 2022-08-24 06:07:02 UTC. -- Aug 24 05:31:53.475000 dv-cinder-854268-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 05:31:53.485578 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 05:31:53 2022] *** Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: nodename: dv-cinder-854268-1-iscsi Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: machine: x86_64 Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: clock source: unix Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: pcre jit disabled Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: detected number of CPU cores: 4 Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: current working directory: / Aug 24 05:31:53.487620 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: detected binary path: /usr/bin/uwsgi-core Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: your processes number limit is 47680 Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: your memory page size is 4096 bytes Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: detected max file descriptor number: 2048 Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: lock engine: pthread robust mutexes Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: thunder lock: enabled Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 05:31:53.488148 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 05:31:53.528949 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: Python main interpreter initialized at 0x55d9dff939c0 Aug 24 05:31:53.528949 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: python threads support enabled Aug 24 05:31:53.528949 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: your server socket listen backlog is limited to 100 connections Aug 24 05:31:53.528949 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: your mercy for graceful operations on workers is 90 seconds Aug 24 05:31:53.529294 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 05:31:53.529294 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: *** Operational MODE: preforking *** Aug 24 05:31:53.529596 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: *** uWSGI is running in multiple interpreter mode *** Aug 24 05:31:53.529596 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: spawned uWSGI master process (pid: 88133) Aug 24 05:31:53.529809 dv-cinder-854268-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 05:31:53.530820 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: spawned uWSGI worker 1 (pid: 88145, cores: 1) Aug 24 05:31:53.530820 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: spawned uWSGI worker 2 (pid: 88146, cores: 1) Aug 24 05:31:53.530820 dv-cinder-854268-1-iscsi devstack@keystone.service[88133]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 05:31:57.859187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:31:57.859187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: from cryptography.utils import int_from_bytes Aug 24 05:31:57.859987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:31:57.859987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: from cryptography.utils import int_from_bytes Aug 24 05:31:57.860228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:31:57.860228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: from cryptography.utils import int_from_bytes Aug 24 05:31:57.860678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 05:31:57.860678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: from cryptography.utils import int_from_bytes Aug 24 05:31:57.908891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 05:31:57.909085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 05:31:57.909256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] command line args: [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 05:31:57.909524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 05:31:57.909743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ================================================================================ {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 05:31:57.909974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] admin_token = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.910156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 05:31:57.910244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] config_dir = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.910354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 05:31:57.910433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.910548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] command line args: [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 05:31:57.910634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] config_source = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.910712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] debug = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.910821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 05:31:57.910898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ================================================================================ {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 05:31:57.911241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] admin_token = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] config_dir = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] config_source = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.911934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] debug = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_config_append = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_dir = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_file = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.912866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_config_append = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.913929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_dir = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_file = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.914925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.915971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.916901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.917019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] publish_errors = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.917675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.917811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.917927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] standard_threads = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] publish_errors = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.918964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] use_journal = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.919902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] use_json = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] use_stderr = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] use_syslog = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] standard_threads = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.920855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.920855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.921075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.921190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] use_journal = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.921308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.921429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] use_json = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.921429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] use_stderr = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.921662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.921789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] use_syslog = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.921912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 05:31:57.922146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.922961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.923908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.924996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.925972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.926974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.927980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.928907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.929963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.930910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.931923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.932985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.933922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.934907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.935977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.936897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.937968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.938885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.939936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.940894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.941997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.942934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.943959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.944883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945359 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.external = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.945959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.password = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] auth.token = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.946941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947198 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.external = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.947910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.password = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] auth.token = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.948987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.949976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.950991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.951941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.952886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.953964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.954961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.955915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.956976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.957923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.958918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.959968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.960907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961747 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.961863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.962979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.963796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.964408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.964527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.964642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.964764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.964881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.964881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.965968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.966913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.967963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.968901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.969977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.970893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.971985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.972914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.973992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.974919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.975943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.976959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.977824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.978897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.979012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] role.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.979135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.979252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] role.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.979366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.979481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.979627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.980267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.980386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.980502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.980617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.980734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.980849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.981999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] role.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] role.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.982929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.983982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.984912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.985974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.986905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.987958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.988901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.989989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990349 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.990935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.991959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.connection = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.992908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.993912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.994884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.connection = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.995974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88145) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 05:31:57.996996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88146) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 05:31:58.023191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.023415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.023635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.024965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.025201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.025431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.032461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.034381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.060822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.061044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.061405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.061662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.115426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.115591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.164581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.164811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.165456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.165718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.322256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.322482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.322704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.322812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.322921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.323045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.323126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.323784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.323880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.324001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.324219 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.324434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.347964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.348736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.359931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.360364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.372629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.372813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.373912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88145) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.374005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.374142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.374342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88146) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 05:31:58.391112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.391320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:31:58.391544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.391662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.391743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:31:58.391862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:31:58.392083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.392170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.392288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 05:31:58.392395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.392569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.392839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.392916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.393159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.393358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.393669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.403089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.403649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.403805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.403959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.404078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.404195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.404314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.404314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.410787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.411251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.413664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.413955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.414161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.414275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.414448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.414577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.414735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.414817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.415935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.416108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.416211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.416481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.416568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.416768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.417068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.439268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.440049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.440710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.440993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.441513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.441826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.445638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.445924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.446210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.446480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.446768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.447059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.453069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.453346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.453940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.454229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.457391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.457707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.457990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.458271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.458351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.458544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.458623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.458817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.458911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.459087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.459168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.459353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.459432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.459670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.459946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.460218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.476581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.476864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.477347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.477660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.480964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.481811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.484171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.484467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.485018 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.485321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.488631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.489515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.491751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.492633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.494994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.495805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.496089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.496384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.496479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.496661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.496741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.497149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.497283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.497283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.497546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.497866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.513176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.513461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.513754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.514027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.514107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.514387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.514663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.514942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.520788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.521804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.522323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.522600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.523326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.523610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.525200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.525487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.525804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.526102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.526250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.526411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.526565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.526865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.527166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.527463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.537650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.537933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.538216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.538501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.538783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.538881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.539062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.539153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.539353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.539443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.539638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.539924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.540203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.540493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.554546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.555777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.557513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.559345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.560903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.562250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.562352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.563688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.563797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.564079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.564374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.565160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.565449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.565779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.570748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.572176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.572262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.574207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.575697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.575986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.576273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.576558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.577200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.577498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.577817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.578101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.584488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.584783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.585065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.585998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.586304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.586594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.591896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88145) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.592177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.592461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.592739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.593022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.593308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.593460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88146) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 05:31:58.593603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.593775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.593876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.594917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.595008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.595193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.595273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.595465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88145) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.595592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.595752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.596033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.596316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.596590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.596860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.597123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88146) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 05:31:58.629684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d9dff939c0 pid: 88145 (default app) Aug 24 05:31:58.631527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d9dff939c0 pid: 88146 (default app) Aug 24 05:31:58.633056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5cbdf-bed0-4f5a-8803-834bcc94665c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:31:58.633341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5cbdf-bed0-4f5a-8803-834bcc94665c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:31:58.633634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5cbdf-bed0-4f5a-8803-834bcc94665c None None] PATH_INFO: `/v3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:31:58.638443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Wed Aug 24 05:31:58 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:05.204275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:05.204879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:05.204879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:05.209134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1/2] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:05.215747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:05.216149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:05.216285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:05.286704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: INFO dbcounter [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] Registered counter for database keystone Aug 24 05:32:05.305689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG oslo_db.sqlalchemy.engines [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88145) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 05:32:05.312383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writer thread running {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 05:32:05.496324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG passlib.handlers.bcrypt [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88145) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 05:32:05.496776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG passlib.handlers.bcrypt [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88145) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 05:32:05.551342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:05.552954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:05.680258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 2/3] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 05:32:06.147890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:06.216885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: INFO dbcounter [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] Registered counter for database keystone Aug 24 05:32:06.238617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG oslo_db.sqlalchemy.engines [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88146) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 05:32:06.244667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writer thread running {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 05:32:06.366023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e619d85-7f61-4755-8a32-93af44c4df57 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:06.370370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:06.372749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:06.373026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:06.373294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:06.381598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:06.439039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 05:32:06.439305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da78362-3a34-4c97-bb3a-53fa0f6fa04f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:06.450062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 2/4] 10.222.0.73 () {58 vars in 1324 bytes} [Wed Aug 24 05:32:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:32:06.456278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:06.471435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bb5b5-ffa6-41cf-b74e-6dc42c463ba3 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:06.475942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:06.477042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:06.477329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:06.477629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] PATH_INFO: `/v3/projects/136c4807ccbf43c49f1d29adcff44403` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:06.479461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] RBAC: Authorizing `identity:get_project(project_id=136c4807ccbf43c49f1d29adcff44403)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:06.532245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING keystone.common.rbac_enforcer.enforcer [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 05:32:06.532596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b04962-df46-4a2f-a6c0-1030736b7fd9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:06.537171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 3/5] 10.222.0.73 () {56 vars in 1207 bytes} [Wed Aug 24 05:32:06 2022] GET /identity/v3/projects/136c4807ccbf43c49f1d29adcff44403 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:08.407187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314fd1-3056-4d84-8ade-bb085ace50b9 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:08.407710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314fd1-3056-4d84-8ade-bb085ace50b9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:08.407802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd314fd1-3056-4d84-8ade-bb085ace50b9 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:08.411882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 3/6] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:08.417262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:08.418183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:08.418424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:08.437934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:08.439458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:08.447556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 4/7] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 24 05:32:08.722348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314fd1-3056-4d84-8ade-bb085ace50b9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:08.734765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd314fd1-3056-4d84-8ade-bb085ace50b9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:08.739009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb1b56-78d7-4429-a42d-b1603d3b6ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:08.739610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1b56-78d7-4429-a42d-b1603d3b6ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:08.739890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1b56-78d7-4429-a42d-b1603d3b6ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:08.740162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb1b56-78d7-4429-a42d-b1603d3b6ad4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:08.747845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb1b56-78d7-4429-a42d-b1603d3b6ad4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:08.749645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb1b56-78d7-4429-a42d-b1603d3b6ad4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:08.758407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 4/8] 10.222.0.73 () {58 vars in 1324 bytes} [Wed Aug 24 05:32:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:32:08.764530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:08.773543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c26e2-ed56-4271-9566-0ec234d7a6d0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:08.779018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beed53-f618-40ed-b41e-c8b5f15f529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:08.779635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beed53-f618-40ed-b41e-c8b5f15f529d admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:08.779915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beed53-f618-40ed-b41e-c8b5f15f529d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:08.780184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beed53-f618-40ed-b41e-c8b5f15f529d admin admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:08.781480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beed53-f618-40ed-b41e-c8b5f15f529d admin admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:08.783408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beed53-f618-40ed-b41e-c8b5f15f529d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:08.784928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 5/9] 10.222.0.73 () {56 vars in 1198 bytes} [Wed Aug 24 05:32:08 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:10.769112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc31fb8-02df-468f-96e2-27a2275def56 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:10.769658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc31fb8-02df-468f-96e2-27a2275def56 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:10.769658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc31fb8-02df-468f-96e2-27a2275def56 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:10.771160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 5/10] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:10.776167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:10.776549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:10.776715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:10.785932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c70a-b78b-419d-b3e9-0925058b3be2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:10.786214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c70a-b78b-419d-b3e9-0925058b3be2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:10.786482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d188c70a-b78b-419d-b3e9-0925058b3be2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:10.787779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 6/11] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:10.793484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:10.793921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:10.794308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:10.795769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:10.796951 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:10.835798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG passlib.handlers.bcrypt [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=88146) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 05:32:10.836232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG passlib.handlers.bcrypt [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88146) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 05:32:10.877254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:10.882820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:10.884465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:10.983074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:11.148250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.172508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.250309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.252920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14211955-2d97-4a16-b7d8-bf7f79238b99 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.289685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.290320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.290601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.290875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.302624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:11.311357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:11.312275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:11.313110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:11.318169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.354440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47c8a5b-d1b3-4a43-8165-27e9085332af None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.357663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.392039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.392897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.393176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.393449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.400228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.420254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62087eeb-7dae-4397-8b5d-374c05ddf22d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.453473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.454130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.454414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.454690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.455372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:11.455684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.457084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.462048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.463558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.469350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 8/15] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:11.475026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.562882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 8/16] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:32:11.568545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.571329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.578344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1446bf1-869e-4511-80d0-ce649004fee8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.613809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.614445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.614727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.615277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.616146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.617633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.619213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 9/17] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:11.624758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.639617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.667580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c92aa3-5c08-42b8-8b33-40d3f9083174 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.701526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.702402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.702679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.702946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.722381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.723851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.725536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40733092-fe33-40d3-a425-ef4506476951 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:32:11.742413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:11.743256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:32:11.744025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:32:11.744025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:32:11.744025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:11.744025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 9/18] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:11.749353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.759450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.760088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.760366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.760637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.762315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.763771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.765529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 10/19] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:11.770970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:11.806729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.851178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:11.851641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3079d1e-a37b-449f-ac2d-516b8dd7844e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.870319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8343150-9b79-4dd4-8923-58fc212818a9 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:11.884991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.885620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.885901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.886650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.887302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:11.887596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.888976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf5c0a-22ca-425f-a450-9a05983ff583 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.903806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:11.904415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:11.904696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:11.904969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:11.905721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.domain.id=default, user.id=a3d3f4f0c68b4b679ab49519fa363f6e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:11.906032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:11.906245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 10/20] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:11.907548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279b50e-836b-440d-8310-ad4d5e3b8003 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:11.917767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 11/21] 10.222.0.73 () {56 vars in 1322 bytes} [Wed Aug 24 05:32:11 2022] GET /identity/v3/role_assignments?user.id=a3d3f4f0c68b4b679ab49519fa363f6e&scope.domain.id=default&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:14.008107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32af20-e29f-4d50-99b7-f725bcc1c5fb None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.009135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32af20-e29f-4d50-99b7-f725bcc1c5fb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.009661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a32af20-e29f-4d50-99b7-f725bcc1c5fb None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.012049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 11/22] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:14.017076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.017506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.017717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.040230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:14.041430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:14.094916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b01153-10af-4c14-9552-87547a717b5c None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.096425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b01153-10af-4c14-9552-87547a717b5c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.096910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b01153-10af-4c14-9552-87547a717b5c None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.098796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 12/23] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:14.104506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c608-4ab2-49c3-8a21-273dcd92e823 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.109380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c608-4ab2-49c3-8a21-273dcd92e823 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.109944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c608-4ab2-49c3-8a21-273dcd92e823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.145701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e7c8c608-4ab2-49c3-8a21-273dcd92e823 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:14.147246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e7c8c608-4ab2-49c3-8a21-273dcd92e823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:14.155970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 12/24] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:14.252657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 13/25] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:14.687009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:14.739818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2813-7be7-4f48-8aee-934af9b3ffb2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.740451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2813-7be7-4f48-8aee-934af9b3ffb2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.740957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2813-7be7-4f48-8aee-934af9b3ffb2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.742790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 14/26] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:14.748453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.749034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.749535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.781806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:14.786272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:14.805928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:14.836458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587d77fe-2da4-42cb-b105-cd8c14a1e1a2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:14.844283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:14.871603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 15/27] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:14.881584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:14.881706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53a1a9-91b3-41d7-8f3c-74386163ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:14.881706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53a1a9-91b3-41d7-8f3c-74386163ecda None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.881706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53a1a9-91b3-41d7-8f3c-74386163ecda None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.881706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53a1a9-91b3-41d7-8f3c-74386163ecda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.889376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53a1a9-91b3-41d7-8f3c-74386163ecda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-ea53a1a9-91b3-41d7-8f3c-74386163ecda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:14.898357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:14.899181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:14.899896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:14.902342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 13/28] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:14.905771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becdd60-00e0-4fde-bfb1-3369443188ad None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:14.907434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becdd60-00e0-4fde-bfb1-3369443188ad None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:14.908553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becdd60-00e0-4fde-bfb1-3369443188ad None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:14.910201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 14/29] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:14.912474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becdd60-00e0-4fde-bfb1-3369443188ad None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.086635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cdd9c1-a5e0-4733-aa85-07eb5d7f7d3d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.119281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becdd60-00e0-4fde-bfb1-3369443188ad None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.174907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.175579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.175863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.176141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.198193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.198848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.199134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.199410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.200098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:15.200408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.201848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56fff31-41b0-4861-8a15-2482ebafee48 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.203399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:15.204314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.206012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.212105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 15/30] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:15.213328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.213798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.214220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:32:15.227444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:15.228766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:32:15.229822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:32:15.229822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:32:15.229822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:15.232064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 16/31] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:15.233109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.238781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:15.240196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:15.268661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:15.327456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 16/32] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:15.328650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.350892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766209c3-e440-4c4e-865f-72ca01937788 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.406092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.407138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.407619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.408113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.408978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.410737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.438174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5e6fa0-b884-4e48-b28f-73689fcd3fa1 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.470074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:15.474738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.475672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.476156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.476625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.478467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.480891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.482739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 17/33] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:15 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:15.483715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.602978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ce700-c0f2-4d81-ad50-7a3f2fb6600c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.619479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 17/34] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 05:32:15 2022] POST /identity/v3/roles => generated 219 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:15.620946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.646278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.646902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.647185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.647458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.648121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:15.648428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.649901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e7499c-f0e4-449f-a4ee-d3ed9916d5f8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.679440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 18/35] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:15.681191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717fe5-7b18-4625-8fea-8ee5f9d56b62 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.681679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717fe5-7b18-4625-8fea-8ee5f9d56b62 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.682106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10717fe5-7b18-4625-8fea-8ee5f9d56b62 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.684316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 19/36] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:15.685273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717fe5-7b18-4625-8fea-8ee5f9d56b62 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.747710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:15.766185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075f684e-ef78-4b41-a695-bc119739d042 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.789703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:15.813767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.814750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.815164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.815570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.817249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.819018 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.820877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 18/37] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:15.821930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:15.850674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10717fe5-7b18-4625-8fea-8ee5f9d56b62 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.888235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.888769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.889046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.889385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.890156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.891552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.928709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ee4c6-2ec2-4bc5-a199-15b28935ddca None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:15.963167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a8a08-4633-4470-84d4-d7364b19a289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:15.963784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a8a08-4633-4470-84d4-d7364b19a289 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:15.964069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a8a08-4633-4470-84d4-d7364b19a289 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:15.964875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a8a08-4633-4470-84d4-d7364b19a289 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:15.965575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a8a08-4633-4470-84d4-d7364b19a289 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:15.966999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a8a08-4633-4470-84d4-d7364b19a289 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:15.995707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 19/38] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:32:15 2022] POST /identity/v3/projects => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:16.009795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09217b60-229e-4bda-92e3-1c0302683fee None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.009795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09217b60-229e-4bda-92e3-1c0302683fee None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.009795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09217b60-229e-4bda-92e3-1c0302683fee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.013059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 20/39] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 05:32:15 2022] POST /identity/v3/roles => generated 225 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:16.014116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:16.019997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-09217b60-229e-4bda-92e3-1c0302683fee None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:16.022144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-09217b60-229e-4bda-92e3-1c0302683fee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:16.025972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:16.132368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 20/40] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:16.134097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4deb6b-0d92-442f-afe0-aba9a9064460 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.134392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4deb6b-0d92-442f-afe0-aba9a9064460 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.134672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4deb6b-0d92-442f-afe0-aba9a9064460 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.141081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 21/41] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:16.144707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.145010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.145278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.172220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:16.173480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:16.255503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 22/42] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:16.270669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58af9e6-d89a-431c-9b9b-16801ef40887 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:16.297789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:16.328804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:16.329485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.329802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.330080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] PATH_INFO: `/v3/domains/default/users/a3d3f4f0c68b4b679ab49519fa363f6e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.332337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a3d3f4f0c68b4b679ab49519fa363f6e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:16.334283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:16.349634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 21/43] 10.222.0.73 () {58 vars in 1361 bytes} [Wed Aug 24 05:32:16 2022] PUT /identity/v3/domains/default/users/a3d3f4f0c68b4b679ab49519fa363f6e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:16.479331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:16.551437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:16.552024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:16.588268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec09a5-4377-422f-9ff0-9f3532bb2676 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:16.625765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:16.626413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.626697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.626974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.628103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:16.629590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:16.631326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 23/44] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:16.637157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:16.678895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0651bb8e-6476-463c-bbcf-0387fba35440 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:16.721060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f04b57-3371-490d-879c-49bfbbe7345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:16.721717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f04b57-3371-490d-879c-49bfbbe7345c None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.722339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f04b57-3371-490d-879c-49bfbbe7345c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.722630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f04b57-3371-490d-879c-49bfbbe7345c None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.723613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f04b57-3371-490d-879c-49bfbbe7345c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:16.725193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f04b57-3371-490d-879c-49bfbbe7345c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:16.741829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 22/45] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 05:32:16 2022] POST /identity/v3/roles => generated 223 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:32:16.751386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42884aa-fe63-4fb1-9366-51e139001172 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:16.791371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3e55b-53e6-4cfe-8d46-c2bb1b2f1c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:16.792020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3e55b-53e6-4cfe-8d46-c2bb1b2f1c0f None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:16.792307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3e55b-53e6-4cfe-8d46-c2bb1b2f1c0f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:16.793122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3e55b-53e6-4cfe-8d46-c2bb1b2f1c0f None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:16.793865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3e55b-53e6-4cfe-8d46-c2bb1b2f1c0f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:16.795389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e3e55b-53e6-4cfe-8d46-c2bb1b2f1c0f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:16.801903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:16.828251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 24/46] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:32:16 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:32:18.304966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ad1a7-9369-4b9a-8011-24a95b522286 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:18.305435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ad1a7-9369-4b9a-8011-24a95b522286 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:18.305541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2ad1a7-9369-4b9a-8011-24a95b522286 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:18.306895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 23/47] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:18.312944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a8e5-c46a-43c7-ac46-7bd73a5ea68a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:18.313355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a8e5-c46a-43c7-ac46-7bd73a5ea68a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:18.313501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a8e5-c46a-43c7-ac46-7bd73a5ea68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:18.331137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-33b0a8e5-c46a-43c7-ac46-7bd73a5ea68a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:18.332333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-33b0a8e5-c46a-43c7-ac46-7bd73a5ea68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:18.400532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 25/48] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:18.675268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ad1a7-9369-4b9a-8011-24a95b522286 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:18.713880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:18.753031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6865-ec99-4ae4-9583-39e665a17299 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:18.753310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6865-ec99-4ae4-9583-39e665a17299 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:18.753601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf6865-ec99-4ae4-9583-39e665a17299 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:18.754930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 26/49] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:18.759884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:18.760168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:18.760448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:18.783621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2ad1a7-9369-4b9a-8011-24a95b522286 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:18.821254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:18.821929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:18.822210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:18.822481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:18.834530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:18.839497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:18.840461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:18.841506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 24/50] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:18.846660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:18.883811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:18.885283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:18.941212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:18.948362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67762f6e-82be-4d6b-8c62-88ed00b92082 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:18.952192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 27/51] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:18.981707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:18.982316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:18.982597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:18.982870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:18.983530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:18.983833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:18.985210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:18.992997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 25/52] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:18.999364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:19.046930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:19.098703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a887ccf-d58d-4bb3-934d-7b7f184f1e4a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:19.132459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:19.133091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:19.133370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:19.133678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:19.135054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:19.136475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:19.138192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 28/53] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:18 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:19.143971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:19.217369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:19.244064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a470b63-ac87-41dd-af77-cd2bb1aed2b1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:19.278123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:19.278755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:19.279035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:19.279302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:19.280475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:19.281966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:19.283547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 26/54] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:19.289307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:19.314626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:19.327094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635458b-4a26-4ac0-bb06-21dd9298bac8 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:19.343613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:19.362400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:19.363042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:19.363327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:19.363607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:19.364723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:19.366230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:19.367803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 29/55] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:19.373860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:19.390634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc6d8bf-ea76-43a7-a30c-06c2341c545e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:19.424024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:19.424619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:19.424959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:19.425233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:19.425981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.domain.id=default, user.id=a3d3f4f0c68b4b679ab49519fa363f6e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:19.426286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:19.427736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d0cb6-ffc3-4daf-88b3-eb63ef450af8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:19.433994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 27/56] 10.222.0.73 () {56 vars in 1322 bytes} [Wed Aug 24 05:32:19 2022] GET /identity/v3/role_assignments?user.id=a3d3f4f0c68b4b679ab49519fa363f6e&scope.domain.id=default&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:19.475814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d249b-f263-4777-a6db-2946b8a19add None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:19.514277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f65f4d1-bb0b-438c-ab9a-0707e0b7e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:19.514899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65f4d1-bb0b-438c-ab9a-0707e0b7e4c6 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:19.515180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65f4d1-bb0b-438c-ab9a-0707e0b7e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:19.515450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f65f4d1-bb0b-438c-ab9a-0707e0b7e4c6 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:19.516111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65f4d1-bb0b-438c-ab9a-0707e0b7e4c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:19.517597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f65f4d1-bb0b-438c-ab9a-0707e0b7e4c6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:19.547249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 30/57] 10.222.0.73 () {60 vars in 1140 bytes} [Wed Aug 24 05:32:19 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:32:21.460683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da70c6-4568-4cb3-97c9-cc7c4e8cb3c9 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:21.461221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da70c6-4568-4cb3-97c9-cc7c4e8cb3c9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:21.461221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da70c6-4568-4cb3-97c9-cc7c4e8cb3c9 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:21.462618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 28/58] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:21.468534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:21.468955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:21.469094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:21.486838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:21.488027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:21.517262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:21.555639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 31/59] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:21.836095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da70c6-4568-4cb3-97c9-cc7c4e8cb3c9 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:21.922827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:21.936051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da70c6-4568-4cb3-97c9-cc7c4e8cb3c9 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:21.969310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3b2e6-e74b-4bb0-a78b-41c287b8a6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:21.969970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3b2e6-e74b-4bb0-a78b-41c287b8a6b6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:21.970251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3b2e6-e74b-4bb0-a78b-41c287b8a6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:21.970527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3b2e6-e74b-4bb0-a78b-41c287b8a6b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:21.971644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3b2e6-e74b-4bb0-a78b-41c287b8a6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:21.973096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3b2e6-e74b-4bb0-a78b-41c287b8a6b6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:21.974689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 29/60] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:21.980440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:22.083291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375fb592-4f48-4aa0-8675-9ae07cea92f6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:22.116506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ac8e2-f19e-473f-90bc-8ee6daa0f062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:22.117058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ac8e2-f19e-473f-90bc-8ee6daa0f062 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:22.117336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ac8e2-f19e-473f-90bc-8ee6daa0f062 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:22.117631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ac8e2-f19e-473f-90bc-8ee6daa0f062 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:22.118284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ac8e2-f19e-473f-90bc-8ee6daa0f062 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:22.119730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ac8e2-f19e-473f-90bc-8ee6daa0f062 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:22.146500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 32/61] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:21 2022] POST /identity/v3/users => generated 275 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:32:24.963523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804aac-f3b0-45c7-9c93-badb80dbeb5d None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:24.964385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804aac-f3b0-45c7-9c93-badb80dbeb5d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:24.964807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78804aac-f3b0-45c7-9c93-badb80dbeb5d None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:24.966328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 30/62] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:24.971767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819bde4-2176-4ea7-8903-ea8c7e168e1b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:24.972157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819bde4-2176-4ea7-8903-ea8c7e168e1b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:24.972323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819bde4-2176-4ea7-8903-ea8c7e168e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:24.996829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f819bde4-2176-4ea7-8903-ea8c7e168e1b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:24.998069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f819bde4-2176-4ea7-8903-ea8c7e168e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:25.019943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:25.078898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 33/63] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:25.381172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804aac-f3b0-45c7-9c93-badb80dbeb5d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:25.415832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f21f2-73c9-4e7a-8dc4-205be506c684 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.416450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f21f2-73c9-4e7a-8dc4-205be506c684 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.416980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f21f2-73c9-4e7a-8dc4-205be506c684 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.418693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 34/64] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:25.423737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11571eb-889b-4da6-88b7-8796865b221f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.424244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11571eb-889b-4da6-88b7-8796865b221f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.424774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11571eb-889b-4da6-88b7-8796865b221f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.455597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e11571eb-889b-4da6-88b7-8796865b221f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:25.458020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e11571eb-889b-4da6-88b7-8796865b221f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:25.529663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78804aac-f3b0-45c7-9c93-badb80dbeb5d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:25.548176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 35/65] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:25.549579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212831f0-e80c-4c52-9818-49bcbd04a4d5 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.550079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212831f0-e80c-4c52-9818-49bcbd04a4d5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.550566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212831f0-e80c-4c52-9818-49bcbd04a4d5 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.552042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 36/66] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:25.570969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e805aa4-83db-4aac-9949-73afcb2e389f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.571583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e805aa4-83db-4aac-9949-73afcb2e389f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.572050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e805aa4-83db-4aac-9949-73afcb2e389f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.578958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:25.579599 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.579879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.580154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.599357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-6e805aa4-83db-4aac-9949-73afcb2e389f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:25.600555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6e805aa4-83db-4aac-9949-73afcb2e389f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:25.609659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:25.615267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:25.616196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:25.616923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 31/67] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:25.624300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:25.662832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:25.739964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 37/68] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:25.786906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fbea9-ffdb-4253-a107-0742436fc1cf None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.787196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fbea9-ffdb-4253-a107-0742436fc1cf None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.787376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fbea9-ffdb-4253-a107-0742436fc1cf None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.788827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 38/69] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:25.794809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.795336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.795798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.815873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:25.817439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:25.829889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:25.832714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e052753-6d8e-45a2-85b4-aef82b7c1b86 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:25.885275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:25.886668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.887085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.887491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.888421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:25.888729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:25.890166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5b72b-1a96-4380-9eb7-e9fe429636f5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:25.899314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 32/70] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:25.900620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb6ec2-f88e-4b6e-bb0b-7cbdc075d0d3 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:25.900907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb6ec2-f88e-4b6e-bb0b-7cbdc075d0d3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:25.901178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb6ec2-f88e-4b6e-bb0b-7cbdc075d0d3 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:25.903012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 33/71] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:25.903940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb6ec2-f88e-4b6e-bb0b-7cbdc075d0d3 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:25.924588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 39/72] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:25.925932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:25.996777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.025240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb6ec2-f88e-4b6e-bb0b-7cbdc075d0d3 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.054345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457c30fd-99ce-4300-bf26-96d3834476bd None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.063707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60ea4f-e048-487e-95f2-499dc37cf420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.064458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60ea4f-e048-487e-95f2-499dc37cf420 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.064750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60ea4f-e048-487e-95f2-499dc37cf420 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.065028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60ea4f-e048-487e-95f2-499dc37cf420 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.076729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60ea4f-e048-487e-95f2-499dc37cf420 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-ea60ea4f-e048-487e-95f2-499dc37cf420 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:26.081891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:26.082726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:26.083490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 34/73] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:26.084240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.084524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.084803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.096974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.098143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.098436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.098717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.103078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:26.104911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:26.114458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.117686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.119446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.121401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 40/74] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:25 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.126520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.176446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 35/75] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:26.177295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.240145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5abf17b-5592-4aae-9cb1-854e875cecbc None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.262165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.287839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.288597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.289160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.289274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.298026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf3b8f1-94e8-486a-bb6b-c0bc676b5650 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.301324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:26.308279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:26.309078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:26.309856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 41/76] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:26.311311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.340358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.340972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.341253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.341527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.342217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:26.342520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.343903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.354093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 36/77] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.358556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.382125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.430177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c972b84f-4198-4807-8629-afad88a2f35c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.466547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cfb94-6003-49c5-99b6-fb4db064083a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.467791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.468398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.468675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.468947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.496097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.502758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.503398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.503679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.503959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.514058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:26.518882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:26.519711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:26.520523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 37/78] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:26.521134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.540585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.542142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.599351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.623099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c988482-4a62-4a19-bb97-8034e424da47 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.646160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 42/79] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:26.646160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.658492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.659106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.659396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.659676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.660345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:26.660652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.662369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.670459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 38/80] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.670459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.724625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.749750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ac4ee-e646-44f5-8075-54ac0b3d8fe6 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.772301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b183e-0d6c-489e-8226-56c4efd8853b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.783701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.784369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.784651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.784926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.786092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.787539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.789107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 43/81] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.789980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.810217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.810841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.811121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.811399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.812517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.814005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.815535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 39/82] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.816288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.820820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.895701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ae1f2-bdbb-48d5-bbe9-34fd15cdb206 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.915206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158014f-9707-4aba-bf46-532b0ac63832 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:26.928103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.928723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.929007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.929284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.929977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:26.930285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.931662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.935956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:26.944101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 44/83] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.944101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:26.956033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:26.956668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:26.956949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:26.957228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:26.958393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:26.959850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:26.961471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 40/84] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:26.962262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.034328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.044590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ce6c34-c91e-4204-acbb-7ae94dc6b766 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.062681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94de84e7-c95d-4369-8598-49532b5e1536 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.079383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.080025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.080308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.080585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.081762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.083223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.084953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 45/85] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:27.085583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.097574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.098208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.098501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.098782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.155095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.169937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.171420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.187600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd5da3-a3ca-4f60-a19e-d1c1ecc73f7f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.220905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.221517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.221847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.222135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.222781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.224231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.247010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 46/86] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:27 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:27.247010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.274801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 41/87] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:27.277168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.277688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.380790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c94012d-4d32-4ccc-a730-771f499adbb9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.381071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.395614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a9a52-c727-42f5-b5ec-ed0189e0e748 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.423681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.424253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.424544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.424830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.426076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.427551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.429144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 47/88] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:27 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:27.430036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.442222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.442841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.443124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.443398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.444141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=f3e3708d9b714f0fa7a56007f70bd68e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:27.444451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.445934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.452932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 42/89] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:27 2022] GET /identity/v3/role_assignments?user.id=f3e3708d9b714f0fa7a56007f70bd68e&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:27.453792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.552745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.578949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2abd72-f4a7-4243-b7c7-5cbce33227b7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.602265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac815a2-fe0d-4862-9eb5-6c024d4fec19 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.628431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.629061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.629347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.629656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.647805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.648854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.649341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.650256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.652775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.655317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.657660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 43/90] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:27 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:27.659342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.692173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.720715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.722902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.788128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a78fe-0f84-43e3-a57b-a11e5917501b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.814265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.835512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.836241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.836773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.836878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.841737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 48/91] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:27.842911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:27.952682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c7087-e30b-4d0c-8c94-f86d2c7c89e1 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:27.960251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.961865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:27.984749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:27.992900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:27.993900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:27.994388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:27.994862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:27.996390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=a3d3f4f0c68b4b679ab49519fa363f6e) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:27.996903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:27.998660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:28.007811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 49/92] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:27 2022] GET /identity/v3/role_assignments?user.id=a3d3f4f0c68b4b679ab49519fa363f6e&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:28.009039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:28.064531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:28.074379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 44/93] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:28.082743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:28.128498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3ea6a-3a91-4a4f-8844-7d4e164f8d2b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:28.169741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:28.171016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:28.171503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:28.172156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:28.174363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:28.176129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:28.179251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 50/94] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:28 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:28.186531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:28.208896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25626d0-af85-4be5-971e-f3da53fea5d5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:28.233151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:28.248945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:28.249647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:28.249957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:28.250387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] PATH_INFO: `/v3/projects/deed56692f1e4b53b91519f82424cf3d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:28.251921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] RBAC: Authorizing `identity:get_project(project_id=deed56692f1e4b53b91519f82424cf3d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:28.253164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:28.255048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 45/95] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:28 2022] GET /identity/v3/projects/deed56692f1e4b53b91519f82424cf3d => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:28.261532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:28.315447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3767f492-fb5b-456f-ab8d-33804f684a4e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:28.331085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:28.363706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:28.364434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:28.364822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:28.365224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:28.366102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] RBAC: Adding query filter params (role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=f3e3708d9b714f0fa7a56007f70bd68e) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:28.366523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:28.368307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea198c9-4223-4ba6-ac9d-a84c9d8c7a0c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:28.376123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ef79d2-b4d5-409c-985a-89a7d96c1b98 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:28.382382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 51/96] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:28 2022] GET /identity/v3/role_assignments?user.id=f3e3708d9b714f0fa7a56007f70bd68e&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:28.415531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:28.416495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:28.416995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:28.417513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:28.418480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=deed56692f1e4b53b91519f82424cf3d, user.id=f3e3708d9b714f0fa7a56007f70bd68e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:28.418997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:28.420716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18220aa6-5bbb-4214-8bf9-02eacb817989 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:28.427861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 46/97] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:28 2022] GET /identity/v3/role_assignments?user.id=f3e3708d9b714f0fa7a56007f70bd68e&scope.project.id=deed56692f1e4b53b91519f82424cf3d&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:30.361801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43d90c-f058-40a9-ae35-7d072dfd44c0 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:30.362769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43d90c-f058-40a9-ae35-7d072dfd44c0 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:30.363244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd43d90c-f058-40a9-ae35-7d072dfd44c0 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:30.364804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 52/98] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:30.370884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e53392-582f-4f61-b4ac-d5c5c375c753 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:30.371961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e53392-582f-4f61-b4ac-d5c5c375c753 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:30.372520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e53392-582f-4f61-b4ac-d5c5c375c753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:30.397056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-02e53392-582f-4f61-b4ac-d5c5c375c753 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:30.398651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-02e53392-582f-4f61-b4ac-d5c5c375c753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:30.458640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:30.483869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 47/99] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:30.600474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea76d7-6922-4f47-8e88-734958a62f5a None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:30.601107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea76d7-6922-4f47-8e88-734958a62f5a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:30.601622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea76d7-6922-4f47-8e88-734958a62f5a None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:30.603170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 53/100] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:30.608820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b0e8-0c5b-4803-ac74-370b218726f1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:30.609112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b0e8-0c5b-4803-ac74-370b218726f1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:30.609393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f4b0e8-0c5b-4803-ac74-370b218726f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:30.638241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e4f4b0e8-0c5b-4803-ac74-370b218726f1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:30.639459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e4f4b0e8-0c5b-4803-ac74-370b218726f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:30.739773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 48/101] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:30.883520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851bd33-2ef4-4d82-8724-1d43b7fb6f6b None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:30.884199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851bd33-2ef4-4d82-8724-1d43b7fb6f6b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:30.884710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851bd33-2ef4-4d82-8724-1d43b7fb6f6b None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:30.886985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 54/102] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:30.892294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:30.892826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:30.893301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:30.922648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:30.924204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:30.926067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851bd33-2ef4-4d82-8724-1d43b7fb6f6b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.072391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 49/103] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:31.090781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851bd33-2ef4-4d82-8724-1d43b7fb6f6b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.095523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.130536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5519c79-af50-40fc-852e-2cdd6f41469a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.132110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519c79-af50-40fc-852e-2cdd6f41469a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.132596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519c79-af50-40fc-852e-2cdd6f41469a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.133066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5519c79-af50-40fc-852e-2cdd6f41469a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.144762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5519c79-af50-40fc-852e-2cdd6f41469a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-e5519c79-af50-40fc-852e-2cdd6f41469a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:31.150436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:31.151248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:31.152073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:31.154424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 55/104] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:31.156730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4691d82-8473-4fc2-ae14-d7234c1023e5 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.157222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4691d82-8473-4fc2-ae14-d7234c1023e5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.157759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4691d82-8473-4fc2-ae14-d7234c1023e5 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.159612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 56/105] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:31.164236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4691d82-8473-4fc2-ae14-d7234c1023e5 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.181909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.216788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.255668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40d8d73-d8c9-4136-ab78-51b7978f4590 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.290200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4691d82-8473-4fc2-ae14-d7234c1023e5 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.295013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9e661-1cd0-4897-ad39-cb336c023f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.295657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9e661-1cd0-4897-ad39-cb336c023f12 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.295941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9e661-1cd0-4897-ad39-cb336c023f12 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.296220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9e661-1cd0-4897-ad39-cb336c023f12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.307094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9e661-1cd0-4897-ad39-cb336c023f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-65c9e661-1cd0-4897-ad39-cb336c023f12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:31.312207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:31.313074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:31.313952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 50/106] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:31.314760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ae427-f8f2-46b3-9ab7-6dbba40c5a63 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.314840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ae427-f8f2-46b3-9ab7-6dbba40c5a63 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.315079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ae427-f8f2-46b3-9ab7-6dbba40c5a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.334803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-675ae427-f8f2-46b3-9ab7-6dbba40c5a63 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:31.335068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.335879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.335999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-675ae427-f8f2-46b3-9ab7-6dbba40c5a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:31.336405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.336825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.337699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:31.338220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:31.340533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91ded6e-2777-468b-a314-42e824ec85c1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:31.349449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 57/107] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:31.350971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93986e19-45e3-4aa8-b778-0e7de23983d1 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.351265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93986e19-45e3-4aa8-b778-0e7de23983d1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.351542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93986e19-45e3-4aa8-b778-0e7de23983d1 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.352868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 58/108] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:31.354100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93986e19-45e3-4aa8-b778-0e7de23983d1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.417658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 51/109] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:31.418873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e2fd-1f7f-4665-8cdb-23c50266923d None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.419171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e2fd-1f7f-4665-8cdb-23c50266923d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.419445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a9e2fd-1f7f-4665-8cdb-23c50266923d None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.421113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 52/110] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:31.425331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e2fd-1f7f-4665-8cdb-23c50266923d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.460580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.460754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.467252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93986e19-45e3-4aa8-b778-0e7de23983d1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.506375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.506995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.507279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.507555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.508223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:31.508529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:31.510009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee63e1-b23d-4ac3-b018-036f9afb648d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:31.524341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 59/111] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:31.524341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c334-9ca9-4466-8ab1-279db1899fd1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.524341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c334-9ca9-4466-8ab1-279db1899fd1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.524341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665c334-9ca9-4466-8ab1-279db1899fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.540565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3665c334-9ca9-4466-8ab1-279db1899fd1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:31.541804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3665c334-9ca9-4466-8ab1-279db1899fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:31.542458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a9e2fd-1f7f-4665-8cdb-23c50266923d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.581476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.582266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.582621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.582971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.584188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:31.585764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:31.587688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 53/112] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:31.588319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.618854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 60/113] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:31.621262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.621553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.621850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.640148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:31.641403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:31.710913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.714192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b18557-1930-41a9-98f2-2f777ace7928 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.715995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 61/114] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:31.716924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.749407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.765191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.766157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.766674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.767146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.779039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:31.785401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:31.786246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:31.786974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:31.789276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 54/115] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:31.795600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.831726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1510a4-aff1-4401-97df-30948d296031 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.867434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.868084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.868727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.868727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] PATH_INFO: `/v3/users/f49a7ac3494745aaa61675ed776347df` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.882678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] RBAC: Authorizing `identity:get_user(user_id=f49a7ac3494745aaa61675ed776347df)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:31.884264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:31.886549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 62/116] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/users/f49a7ac3494745aaa61675ed776347df => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:31.887406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.906820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0db7ba-c448-40c4-a735-3bb8f0880891 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:31.939318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:31.939941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:31.940218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:31.940496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:31.941634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:31.943099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:31.944721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 55/117] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:31.945477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:31.978718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:31.981322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.002274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6097dc50-761d-46eb-b333-1582961e96a9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.036977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.037641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.037926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.038209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.039829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.040819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.042478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 63/118] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:32.043304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.067225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16894a34-dfcc-45db-a325-37c1c3b503a2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.100916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.101519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.101827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.102099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.102750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:32.103051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.104714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.112414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 56/119] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:32:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:32.114440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.142687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eece92-8f0b-4bbf-92cd-21aacc755e30 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.178455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.179068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.179348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.179621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.197272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.204792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.215206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939957e3-189b-417f-b62b-2f1578545777 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.247279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.248684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.249933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.250650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.250937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.251218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.264071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:32.269583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:32.270548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 57/120] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:32.271786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.366187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 64/121] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:32.367097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.383814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66e454c-3183-49bf-9a0c-1d9da2dd5fb1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.419028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.419692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.419965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.420243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/f3e3708d9b714f0fa7a56007f70bd68e/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.421549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.422203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] RBAC: Authorizing `identity:create_grant(project_id=68f16fe3336647ac92f5dc9a178bc2d4, user_id=f3e3708d9b714f0fa7a56007f70bd68e, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.423741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.436551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 58/122] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:32 2022] PUT /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/f3e3708d9b714f0fa7a56007f70bd68e/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:32.437194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.459435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.478713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb4d72-bacb-49ae-b0c8-ad4d235c25de None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.521713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.522387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.522736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.523017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.535650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:32.540627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:32.541642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:32.543014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 65/123] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:32.543620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.569700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b17b83-d29a-41ad-8236-2b6b54d306ca None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.614588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.615209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.615494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.615771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.616519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.618139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.649547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 59/124] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:32:32 2022] POST /identity/v3/projects => generated 302 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:32.653604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.664876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c24d1-4afc-4a22-80b5-ccf9bf9bbd6b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.678727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.712132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.712906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.713264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.713690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.715068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.716998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.718889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 66/125] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:32.719758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.743980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:32.803744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7480b50d-5420-44d6-be77-e837d8063904 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.839029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8ce9f-4424-4d1d-b353-f1bbc466451c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:32.852582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.852890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.853145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.853707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.857215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:32.857215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.858090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.868730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 60/126] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:32.871328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.881959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:32.883026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:32.883312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:32.883589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:32.885086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:32.887267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:32.889479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 67/127] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:32.890315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:32.949179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:33.004350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185f1e0-485a-4bb6-93c2-7b52460ea0e7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.024233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9920ff-1a44-43d7-9c7b-a73d59c6dc6e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.048165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:33.051415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.052040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.052640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.052928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.054162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:33.054905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.057176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.062047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.062887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.063268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.063637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.064927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.066584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.067561 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 61/128] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.068400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 68/129] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:32 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.072615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.074467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.187521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa12ec4-80ae-4c77-a3e0-2772651d7998 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.200680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:33.205552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8467592-4f33-4910-b180-7b12efa36dc6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.225211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.226492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.226986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.227386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.228235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] RBAC: Adding query filter params (role.id=14254fae7ef74b73870f800cbc43540a, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=f49a7ac3494745aaa61675ed776347df) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:33.228894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.231073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.237129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 69/130] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:33 2022] GET /identity/v3/role_assignments?user.id=f49a7ac3494745aaa61675ed776347df&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=14254fae7ef74b73870f800cbc43540a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.238099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.272863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.272863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.272863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.272863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.276881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.276881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.278423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 62/131] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:33 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.279377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.417130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cbc91-6161-4a30-8baa-b87269d9af19 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.420551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:33.463703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.464998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.465619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.465826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/f3e3708d9b714f0fa7a56007f70bd68e/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.473164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f16fe3336647ac92f5dc9a178bc2d4, user_id=f3e3708d9b714f0fa7a56007f70bd68e, role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.474645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.475738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac2f79d-37a4-4ad9-82c7-8aafb97e3faf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.507688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 70/132] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:33 2022] PUT /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/f3e3708d9b714f0fa7a56007f70bd68e/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:33.508602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.513813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.514827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.515311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.515786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.517365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.519305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.521235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 63/133] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:33 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.530593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.615742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:33.773294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8502c8-fa53-409f-9972-bab5abcca2b7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.797496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a0c68-1d77-47ca-965b-b600ae86a207 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:33.826054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:33.829730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.830516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.830795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.831068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.841588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.841588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.842400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 71/134] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:33 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.848475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:33.849351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:33.850273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:33.850751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:33.851216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] PATH_INFO: `/v3/projects/deed56692f1e4b53b91519f82424cf3d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:33.852588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] RBAC: Authorizing `identity:get_project(project_id=deed56692f1e4b53b91519f82424cf3d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:33.854738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:33.856769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 64/135] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:33 2022] GET /identity/v3/projects/deed56692f1e4b53b91519f82424cf3d => generated 312 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:33.862440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:34.022143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98728577-9922-4053-9d53-65d4632f9078 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:34.036734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93430a8-8706-4c2c-aa46-979bebe9f3ee None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:34.060229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:34.094307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ad8ce-990a-4985-a3ee-07eac9eb1071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:34.095001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ad8ce-990a-4985-a3ee-07eac9eb1071 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:34.095284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ad8ce-990a-4985-a3ee-07eac9eb1071 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:34.095561 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ad8ce-990a-4985-a3ee-07eac9eb1071 None admin] PATH_INFO: `/v3/projects/deed56692f1e4b53b91519f82424cf3d/users/f3e3708d9b714f0fa7a56007f70bd68e/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:34.097593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ad8ce-990a-4985-a3ee-07eac9eb1071 None admin] RBAC: Authorizing `identity:create_grant(project_id=deed56692f1e4b53b91519f82424cf3d, user_id=f3e3708d9b714f0fa7a56007f70bd68e, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:34.099062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ad8ce-990a-4985-a3ee-07eac9eb1071 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:34.124240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 65/136] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:33 2022] PUT /identity/v3/projects/deed56692f1e4b53b91519f82424cf3d/users/f3e3708d9b714f0fa7a56007f70bd68e/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:32:34.126508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b061004-ee11-4ba5-b68c-d9ecac42570b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:34.127198 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b061004-ee11-4ba5-b68c-d9ecac42570b None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:34.127497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b061004-ee11-4ba5-b68c-d9ecac42570b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:34.127771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b061004-ee11-4ba5-b68c-d9ecac42570b None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/a3d3f4f0c68b4b679ab49519fa363f6e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:34.135334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b061004-ee11-4ba5-b68c-d9ecac42570b None admin] RBAC: Authorizing `identity:create_grant(project_id=68f16fe3336647ac92f5dc9a178bc2d4, user_id=a3d3f4f0c68b4b679ab49519fa363f6e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:34.136894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b061004-ee11-4ba5-b68c-d9ecac42570b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:34.158052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 72/137] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:33 2022] PUT /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/a3d3f4f0c68b4b679ab49519fa363f6e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:32:34.657713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27b78-86f2-449f-ab0a-aeb1ae46b263 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:34.658877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27b78-86f2-449f-ab0a-aeb1ae46b263 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:34.659362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e27b78-86f2-449f-ab0a-aeb1ae46b263 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:34.661175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 66/138] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:34.667050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9d0a-ebd8-4e1b-a8d8-41bdc09591ec None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:34.668109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9d0a-ebd8-4e1b-a8d8-41bdc09591ec None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:34.668752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c9d0a-ebd8-4e1b-a8d8-41bdc09591ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:34.694926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c88c9d0a-ebd8-4e1b-a8d8-41bdc09591ec None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:34.696486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c88c9d0a-ebd8-4e1b-a8d8-41bdc09591ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:34.800118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 73/139] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:34.962829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0ebc-a913-4d72-9b65-fe9d66cce3bb None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:34.963141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0ebc-a913-4d72-9b65-fe9d66cce3bb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:34.963350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f0ebc-a913-4d72-9b65-fe9d66cce3bb None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:34.964803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 67/140] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:34.970130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:34.970671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:34.971142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:34.994184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:34.996265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:35.083438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 74/141] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:35.235814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0ebc-a913-4d72-9b65-fe9d66cce3bb None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:35.303234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:35.355934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f0ebc-a913-4d72-9b65-fe9d66cce3bb None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:35.390899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:35.391533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:35.391816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:35.392083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:35.402940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:35.408217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:35.409041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:35.410179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 68/142] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:35.415131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:35.440318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:35.518305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73586bc0-2001-4444-b229-05dcbbfeb3fb None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:35.560858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:35.561493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:35.561833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:35.562088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:35.562829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:35.563142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:35.564538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:35.574013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 75/143] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:35.579637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:35.621268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:35.753965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566904d-402d-42f4-9699-0791e5d96b71 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:35.775490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:35.782702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:35.811959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b02e1-08f9-412d-ac91-507eecebb823 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:35.826393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:35.827091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:35.827368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:35.827685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:35.830028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:35.831493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:35.833849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 69/144] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:35 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:35.848383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:35.884971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:35.885651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:35.885983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:35.886258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:35.888356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:35.889852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:35.891686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 76/145] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:35.897084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:35.989051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd35317-e043-4c29-a3d1-952b732461f4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:36.043888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbd4913-ed37-422f-b999-e4ed8d4e4528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:36.044483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4913-ed37-422f-b999-e4ed8d4e4528 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.044769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4913-ed37-422f-b999-e4ed8d4e4528 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.045044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbd4913-ed37-422f-b999-e4ed8d4e4528 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.111596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04216b6-d403-4c88-a821-bca72f2bd946 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:36.141928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:36.178400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7a9a5-d844-424b-9e98-670e67fde9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:36.179023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7a9a5-d844-424b-9e98-670e67fde9f7 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.179303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7a9a5-d844-424b-9e98-670e67fde9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.179572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7a9a5-d844-424b-9e98-670e67fde9f7 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.180229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7a9a5-d844-424b-9e98-670e67fde9f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:36.181732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7a9a5-d844-424b-9e98-670e67fde9f7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:36.186970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd4913-ed37-422f-b999-e4ed8d4e4528 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:36.188462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbd4913-ed37-422f-b999-e4ed8d4e4528 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:36.236696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 77/146] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:35 2022] POST /identity/v3/users => generated 283 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:36.306653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc48d72-a1df-471c-bd63-91243d5e6b96 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.306940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc48d72-a1df-471c-bd63-91243d5e6b96 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.307124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc48d72-a1df-471c-bd63-91243d5e6b96 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.308511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 78/147] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:36.314187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8e12-cdd7-4dce-9939-9a7dc072664a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.314492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8e12-cdd7-4dce-9939-9a7dc072664a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.314731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d8e12-cdd7-4dce-9939-9a7dc072664a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.344095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-972d8e12-cdd7-4dce-9939-9a7dc072664a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:36.347593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-972d8e12-cdd7-4dce-9939-9a7dc072664a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:36.355417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:36.357324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 70/148] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:32:36.357943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bc7fc-1eb6-4caa-a406-10aa2adde8bd None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.358892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bc7fc-1eb6-4caa-a406-10aa2adde8bd None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.358892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18bc7fc-1eb6-4caa-a406-10aa2adde8bd None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.360039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 71/149] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:36.367524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bc7fc-1eb6-4caa-a406-10aa2adde8bd None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:36.450949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 79/150] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:36.452808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.453356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.453868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.480973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:36.487230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:36.568295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:36.593500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18bc7fc-1eb6-4caa-a406-10aa2adde8bd None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:36.675409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 80/151] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:36.677874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:36.678753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.679245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.679712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.681037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:36.683242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:36.685555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 72/152] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:36 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:36.693068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:36.780606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:36.807114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:36.828816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58847c-8a09-4a76-8413-8729f9ab83e1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:36.894209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:36.894832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.895110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.895383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:36.896122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=f3e3708d9b714f0fa7a56007f70bd68e) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:36.896430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:36.897919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:36.913806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 81/153] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:36 2022] GET /identity/v3/role_assignments?user.id=f3e3708d9b714f0fa7a56007f70bd68e&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:36.960681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173cb2ac-e014-4810-bd1f-fb8f4c8ac16c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:36.997967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:36.998624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:36.998898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:36.999171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.012624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:37.017607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:37.018532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:37.019704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 73/154] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:32:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:37.024932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.138748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee334528-c495-424f-8a78-168fb7cc98c6 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:37.177179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.184250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:37.184894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.185178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.185452 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.186166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:37.186470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:37.187868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:37.188053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:37.189910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:37.207790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 82/155] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:37.212911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.312673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0025e95c-89e4-43bc-8186-2be7f10cb7e1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:37.352660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72da53c2-4a42-499e-97fe-7e51a5c5ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:37.353543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da53c2-4a42-499e-97fe-7e51a5c5ba0c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.353856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da53c2-4a42-499e-97fe-7e51a5c5ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.354120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72da53c2-4a42-499e-97fe-7e51a5c5ba0c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.365813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72da53c2-4a42-499e-97fe-7e51a5c5ba0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-72da53c2-4a42-499e-97fe-7e51a5c5ba0c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:37.370707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:37.371542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 74/156] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:37.373635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254978a-34eb-4dd1-96c2-de16d319ea5c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.375029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254978a-34eb-4dd1-96c2-de16d319ea5c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.375029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a254978a-34eb-4dd1-96c2-de16d319ea5c None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.375597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 75/157] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:37.377344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422d1e9-cdfa-4ab0-a70a-9b5e65df80af None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.377344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422d1e9-cdfa-4ab0-a70a-9b5e65df80af None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.377344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422d1e9-cdfa-4ab0-a70a-9b5e65df80af None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.383319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 76/158] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:37.384086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422d1e9-cdfa-4ab0-a70a-9b5e65df80af None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.395640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5735875-e289-4729-95e8-2555d6a93d7c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:37.447824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c180e4-b636-4f26-9ca4-c55a82c5b56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:37.449618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c180e4-b636-4f26-9ca4-c55a82c5b56a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.450121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c180e4-b636-4f26-9ca4-c55a82c5b56a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.450584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c180e4-b636-4f26-9ca4-c55a82c5b56a None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.452476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c180e4-b636-4f26-9ca4-c55a82c5b56a None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:37.454638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c180e4-b636-4f26-9ca4-c55a82c5b56a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:37.457217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 83/159] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:37.459381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.460034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.460810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.464653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:37.489298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:37.491450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:37.511484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422d1e9-cdfa-4ab0-a70a-9b5e65df80af None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:37.563422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:37.564493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.565201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.565674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.566532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:37.567002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:37.569153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3db2d-bb70-4b32-ae61-4f6aa6458e84 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:37.576149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:37.581128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 77/160] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:37.583032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.583312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.583805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.583923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 84/161] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:37.584498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.610784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:37.611980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:37.710413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 78/162] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:37.713182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.735558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb425b-15c4-49de-82f1-ee3edc4656ce None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:37.812279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:37.830333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:37.830989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.831244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.831517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.877929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f9cf8a-8e96-4d44-a88f-d74218732b58 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:37.938375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:37.939385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:37.939887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:37.940361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] PATH_INFO: `/v3/users/f49a7ac3494745aaa61675ed776347df` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:37.941850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] RBAC: Authorizing `identity:get_user(user_id=f49a7ac3494745aaa61675ed776347df)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:37.943538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:37.945349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 79/163] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/users/f49a7ac3494745aaa61675ed776347df => generated 289 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:37.952476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:37.976939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:37.994675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:37.996150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:38.096937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95e3fa-0dd4-4f09-93f1-ad774c9c52bb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:38.153924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:38.172568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:38.173210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:38.173486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:38.173796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:38.175054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:38.176514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:38.178578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 80/164] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:38.179348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:38.244366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 85/165] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:38.245589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:38.324847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3a1a2-3562-4c90-9f39-87d2ce93bc38 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:38.373132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:38.381310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:38.383004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:38.383712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:38.386262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011b0c90-7f17-42b9-8799-a58d59aa43e4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:38.386755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:38.410315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:38.415408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:38.416245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:38.417552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:38.425884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 81/166] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:38.427899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:38.466980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:38.467667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:38.467992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:38.468266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/f49a7ac3494745aaa61675ed776347df/roles/14254fae7ef74b73870f800cbc43540a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:38.473002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f16fe3336647ac92f5dc9a178bc2d4, user_id=f49a7ac3494745aaa61675ed776347df, role_id=14254fae7ef74b73870f800cbc43540a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:38.474585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:38.516001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:38.516220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 86/167] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:38 2022] PUT /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/users/f49a7ac3494745aaa61675ed776347df/roles/14254fae7ef74b73870f800cbc43540a => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:38.516220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:38.608468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9704cc8-0620-49b5-a326-764edf009839 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:38.674783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:38.675827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:38.676276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:38.676689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:38.698671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:38.703679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:38.704582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:38.705337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 82/168] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:38.713780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:38.779739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c820a48-24cf-4c6c-b320-7a5633177537 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:38.852571 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:38.853596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:38.853792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:38.854028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:38.858012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:38.859555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:38.861401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 87/169] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:38 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:38.862020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:38.912402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a66630-be0f-46c9-9c05-5947bd1a27b0 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:38.921938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:38.947428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:38.974032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:38.975042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:38.975521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:38.975990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:38.976847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:38.977349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:38.979265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:38.989033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 83/170] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:38.990200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:39.061025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53082f73-1139-46f0-a977-57528cdb1228 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:39.095410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8965a-98ec-4b35-b3ce-fcbc60a86835 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:39.107291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:39.107923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.108216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.108486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.109166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:39.109471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:39.110999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:39.129725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 88/171] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:39.130589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:39.133433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:39.134406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.134874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.135355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.137013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] RBAC: Adding query filter params (role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=f3e3708d9b714f0fa7a56007f70bd68e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:39.137588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:39.139271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:39.146120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 84/172] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:38 2022] GET /identity/v3/role_assignments?user.id=f3e3708d9b714f0fa7a56007f70bd68e&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:39.147148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:39.194822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:39.260570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620fc2-29f1-43d6-9e76-a1aab01d7e81 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:39.299350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:39.300399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.300940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.301401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] PATH_INFO: `/v3/users/f3e3708d9b714f0fa7a56007f70bd68e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.302906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] RBAC: Authorizing `identity:get_user(user_id=f3e3708d9b714f0fa7a56007f70bd68e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:39.304585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:39.305975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:39.306716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 85/173] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:39 2022] GET /identity/v3/users/f3e3708d9b714f0fa7a56007f70bd68e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:39.314165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:39.347522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a99805-2b66-4db2-a671-0a0f59934524 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:39.428370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:39.429422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.429921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.430397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] PATH_INFO: `/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.432548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] RBAC: Authorizing `identity:get_user(user_id=a3d3f4f0c68b4b679ab49519fa363f6e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:39.437203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:39.439281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 89/174] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:39 2022] GET /identity/v3/users/a3d3f4f0c68b4b679ab49519fa363f6e => generated 247 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:39.445669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:39.508182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72179a90-bab6-4170-832a-548a91f4b0bf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:39.519687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:39.553475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c562a-0a4b-4740-a64d-82e3fa73ed1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:39.554506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c562a-0a4b-4740-a64d-82e3fa73ed1f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.554791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c562a-0a4b-4740-a64d-82e3fa73ed1f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.555064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c562a-0a4b-4740-a64d-82e3fa73ed1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.640950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70f9a64-9cf6-4312-bfb2-50b41e72c42e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:39.679330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c562a-0a4b-4740-a64d-82e3fa73ed1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:39.681257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c562a-0a4b-4740-a64d-82e3fa73ed1f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:39.700829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb7bfd-6b93-4beb-b7b8-153f1166be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:39.701850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb7bfd-6b93-4beb-b7b8-153f1166be73 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.702338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb7bfd-6b93-4beb-b7b8-153f1166be73 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.702796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb7bfd-6b93-4beb-b7b8-153f1166be73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.744648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:39.829832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb7bfd-6b93-4beb-b7b8-153f1166be73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:39.831657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb7bfd-6b93-4beb-b7b8-153f1166be73 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:39.875480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 86/175] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:39.877089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf932-d2a5-41fd-b083-37b558a0b168 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:39.877783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf932-d2a5-41fd-b083-37b558a0b168 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:39.879442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf932-d2a5-41fd-b083-37b558a0b168 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:39.881646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 87/176] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:39.882581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf932-d2a5-41fd-b083-37b558a0b168 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:39.907858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:40.019941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 90/177] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:40.021123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.021466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.021768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.053851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf932-d2a5-41fd-b083-37b558a0b168 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:40.066264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:40.067818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:40.115920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:40.117635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.118263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.119709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] PATH_INFO: `/v3/projects/deed56692f1e4b53b91519f82424cf3d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.123125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] RBAC: Authorizing `identity:get_project(project_id=deed56692f1e4b53b91519f82424cf3d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:40.126969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:40.129056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 88/178] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:39 2022] GET /identity/v3/projects/deed56692f1e4b53b91519f82424cf3d => generated 312 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:40.131395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:40.161439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:40.200705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 91/179] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:40.203417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:40.277866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:40.318343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9338124-0efb-49e4-9c0c-642bf7d29329 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:40.383176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff0967-6c3c-480f-8b7c-a3e7b0924317 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:40.402298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f443e-6616-4422-9d68-58adee4f7a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:40.402957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f443e-6616-4422-9d68-58adee4f7a53 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.403241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f443e-6616-4422-9d68-58adee4f7a53 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.403512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f443e-6616-4422-9d68-58adee4f7a53 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.410282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f443e-6616-4422-9d68-58adee4f7a53 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:40.411836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f443e-6616-4422-9d68-58adee4f7a53 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:40.413668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 89/180] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:40 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:40.414612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d951c8d-4ab7-40c8-bfae-4e66ee7bc601 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.414891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d951c8d-4ab7-40c8-bfae-4e66ee7bc601 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.415164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d951c8d-4ab7-40c8-bfae-4e66ee7bc601 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.416608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 90/181] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:40.421523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d951c8d-4ab7-40c8-bfae-4e66ee7bc601 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:40.427273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:40.428125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.428893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.430040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.430923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=deed56692f1e4b53b91519f82424cf3d, user.id=f3e3708d9b714f0fa7a56007f70bd68e) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:40.433928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:40.435556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2e2f53-0efe-499e-9c1b-91a551784ff4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:40.450163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 92/182] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:40 2022] GET /identity/v3/role_assignments?user.id=f3e3708d9b714f0fa7a56007f70bd68e&scope.project.id=deed56692f1e4b53b91519f82424cf3d&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:40.451628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c81ce-3091-4d52-9e61-7fbb4ff6c448 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.454971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c81ce-3091-4d52-9e61-7fbb4ff6c448 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.455581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4c81ce-3091-4d52-9e61-7fbb4ff6c448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.485962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-da4c81ce-3091-4d52-9e61-7fbb4ff6c448 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:40.490914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-da4c81ce-3091-4d52-9e61-7fbb4ff6c448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:40.571937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:40.607454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 93/183] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:40.609121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e907-3782-4363-8d61-ef5ad105b565 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.609605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e907-3782-4363-8d61-ef5ad105b565 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.609973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7862e907-3782-4363-8d61-ef5ad105b565 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.611041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d951c8d-4ab7-40c8-bfae-4e66ee7bc601 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:40.611773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 94/184] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:40.612621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e907-3782-4363-8d61-ef5ad105b565 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:40.680654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:40.681192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.681472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.681780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.683065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=a3d3f4f0c68b4b679ab49519fa363f6e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:40.683377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:40.684927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b3d85-6a6d-414d-92f6-8824169d4bc5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:40.705700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 91/185] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:40 2022] GET /identity/v3/role_assignments?user.id=a3d3f4f0c68b4b679ab49519fa363f6e&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:40.707159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.707449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.707725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.746320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:40.748103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:40.753827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:40.829447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7862e907-3782-4363-8d61-ef5ad105b565 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:40.889241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 92/186] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:40.906372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:40.907030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:40.907307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:40.907574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:40.920960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:40.926211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:40.927100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:40.927857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:40.930376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 95/187] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:40.937699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:40.954599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.012282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.077459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25fb775-122e-4ab2-9f9f-8836c42d7f40 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.085783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568f24d5-a8d3-4bca-8e1f-0e5d3aacdc0d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.116204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.116814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.117088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.117361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.120618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:41.120618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.120618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52502f00-e018-4717-9058-76a6706cb070 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.125805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.127259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb372f-7cbc-435d-a2e4-ed607abede3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.127911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb372f-7cbc-435d-a2e4-ed607abede3c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.128189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb372f-7cbc-435d-a2e4-ed607abede3c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.128461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb372f-7cbc-435d-a2e4-ed607abede3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.129146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 93/188] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:41.135019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba3c5a-5290-45dd-bc0c-394ba4ca99c2 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.136100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba3c5a-5290-45dd-bc0c-394ba4ca99c2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.136719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ba3c5a-5290-45dd-bc0c-394ba4ca99c2 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.140441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb372f-7cbc-435d-a2e4-ed607abede3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:41.140717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 94/189] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:41 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:41.141759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba3c5a-5290-45dd-bc0c-394ba4ca99c2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-21bb372f-7cbc-435d-a2e4-ed607abede3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:41.145245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:41.146108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:41.146871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 96/190] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:32:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:41.147596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.147800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.148070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.166894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:41.168426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:41.236325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 97/191] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:41.237080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.264581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ba3c5a-5290-45dd-bc0c-394ba4ca99c2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.272673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.304179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.304821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.305101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.305374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] PATH_INFO: `/v3/users/535f7ae859e047bb869814b1ed485184` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.320625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] RBAC: Authorizing `identity:get_user(user_id=535f7ae859e047bb869814b1ed485184)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.322142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.324335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 95/192] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/users/535f7ae859e047bb869814b1ed485184 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:41.325110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.354498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5d6cd-dd71-4191-ae95-4fb5d8aa373c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.389990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.391196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.391869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.392232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.392546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.393302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:41.393631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.394997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.405288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 98/193] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:41.406358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.446094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01993d42-60f9-467b-aa64-02f277531a37 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.483152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.483791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.484071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.484343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.485593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.487231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.489032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 96/194] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:41.494408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.495868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.509071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316ed703-a9dc-4c5c-b90f-b467bd45283c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.543573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.544196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.544477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.544757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.597590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000971d5-103c-4f35-bf66-01f32f1e0b1c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.614060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.615570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.623383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.632036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.632688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.632962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.633232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] PATH_INFO: `/v3/users/535f7ae859e047bb869814b1ed485184` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.634378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] RBAC: Authorizing `identity:get_user(user_id=535f7ae859e047bb869814b1ed485184)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.636360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.638082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 97/195] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/users/535f7ae859e047bb869814b1ed485184 => generated 283 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:41.638702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.705975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.719051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 99/196] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:41.724393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.736287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1297f6a3-8109-49c1-a8af-47060f5ae0f4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.769634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.770173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.770445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.770713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.771348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:41.772783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:41.799199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 98/197] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:41 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:41.800022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.833142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d318938b-302f-4e76-8d75-63bc78fb36ec None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.873063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.873879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.874227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.874508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:41.876645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.887946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:41.892904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:41.893919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:41.894675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 100/198] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:41.895315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:41.945836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c21679-7f9b-441c-9a01-b3f1355e6e3b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:41.951952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:41.986217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:41.986872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:41.987159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:41.987434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.018424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8320b-63cd-40d2-8944-c83234de007a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.058031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.059684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.059958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.060231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.061497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.063003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.064834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.065117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 101/199] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:42.066677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.066794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:42.141407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:42.200659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 99/200] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:42.201593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:42.205891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b592a0f-4609-475d-bf06-aed469e58254 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.248917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.250282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.250679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.251053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.252241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:42.252686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.255775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.257908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:42.276928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 102/201] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:42.279746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:42.364033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eda99-1d51-4592-86fd-5d8725a1b5b1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.403654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.404304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.404561 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.404830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.405636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=535f7ae859e047bb869814b1ed485184) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:42.405965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.407440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.417602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 100/202] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:42 2022] GET /identity/v3/role_assignments?user.id=535f7ae859e047bb869814b1ed485184&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:42.417602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:42.441512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c18615-7fe5-49cc-862c-2dac02eab075 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.451104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:42.505974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1232067-36af-447c-923b-c14660de13cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.506907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1232067-36af-447c-923b-c14660de13cc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.507196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1232067-36af-447c-923b-c14660de13cc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.507467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1232067-36af-447c-923b-c14660de13cc None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.508871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1232067-36af-447c-923b-c14660de13cc None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.510575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1232067-36af-447c-923b-c14660de13cc None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.512842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 103/203] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:42 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:42.520414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1232067-36af-447c-923b-c14660de13cc None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:42.583738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:42.627063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bfaec-c3eb-4f28-8592-7c4fd00a2587 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.662969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.663709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.663987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.664274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] PATH_INFO: `/v3/users/f49a7ac3494745aaa61675ed776347df` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.665456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] RBAC: Authorizing `identity:get_user(user_id=f49a7ac3494745aaa61675ed776347df)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.667592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.669690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 101/204] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:42 2022] GET /identity/v3/users/f49a7ac3494745aaa61675ed776347df => generated 289 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:42.674013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1232067-36af-447c-923b-c14660de13cc None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.675453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:42.737877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.738887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.739272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.739647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.740486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] RBAC: Adding query filter params (role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=535f7ae859e047bb869814b1ed485184) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:42.740996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.742796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:42.756468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 104/205] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:42 2022] GET /identity/v3/role_assignments?user.id=535f7ae859e047bb869814b1ed485184&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:42.765980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:42.783618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec9389-1a97-40f7-ad6c-8bcd51bf0b50 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:42.826072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:42.826696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:42.826972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:42.827243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:42.912874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:42.914299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:43.003885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:43.029785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 102/206] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:43.031683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:43.151886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2d5cc7-ea6f-43d9-8f32-b5b3ac4aa27e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:43.167724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:43.202778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226b8aa-eacc-47d5-b691-7db41d483fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:43.203428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226b8aa-eacc-47d5-b691-7db41d483fe5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:43.203710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226b8aa-eacc-47d5-b691-7db41d483fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:43.203985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226b8aa-eacc-47d5-b691-7db41d483fe5 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:43.210297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226b8aa-eacc-47d5-b691-7db41d483fe5 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:43.211826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f226b8aa-eacc-47d5-b691-7db41d483fe5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:43.213518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 105/207] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:43 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:43.222620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:43.369760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370753e1-a15b-410d-8b54-d932066dceb8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:43.416590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:43.417598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:43.418137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:43.418625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:43.419568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] RBAC: Adding query filter params (role.id=14254fae7ef74b73870f800cbc43540a, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4, user.id=f49a7ac3494745aaa61675ed776347df) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:43.420090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:43.421757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f81b10-8bef-440f-81b1-085f4abb23ec None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:43.428244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 103/208] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:43 2022] GET /identity/v3/role_assignments?user.id=f49a7ac3494745aaa61675ed776347df&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=14254fae7ef74b73870f800cbc43540a => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:43.971220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4a7d7-f7f1-469b-aed5-2923b64324c2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:43.971852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4a7d7-f7f1-469b-aed5-2923b64324c2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:43.972333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4a7d7-f7f1-469b-aed5-2923b64324c2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:43.974171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 106/209] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:43.982430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574f567-034a-4da2-a638-2374378efb9a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:43.982719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574f567-034a-4da2-a638-2374378efb9a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:43.982995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574f567-034a-4da2-a638-2374378efb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.007471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a574f567-034a-4da2-a638-2374378efb9a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:44.008671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a574f567-034a-4da2-a638-2374378efb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:44.060764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:44.090619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 104/210] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:44.281366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331cb79-1058-486c-9314-3522514c057f None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.281872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331cb79-1058-486c-9314-3522514c057f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.281945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f331cb79-1058-486c-9314-3522514c057f None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.283427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 107/211] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:44.288979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.289859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.290355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.309871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:44.311390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:44.382727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 105/212] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:44.387152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331cb79-1058-486c-9314-3522514c057f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:44.493675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f331cb79-1058-486c-9314-3522514c057f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:44.529772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:44.530333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.530610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.530881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.541271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:44.546637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:44.547499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:44.548257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 108/213] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:44.553808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:44.664191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db19f212-eca8-43e1-bf66-d6cdd77fc896 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:44.672682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:44.693913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:44.702722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:44.703345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.703625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.703895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.704567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:44.704866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:44.706814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:44.718353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 106/214] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:44.723747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:44.743016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:44.813095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7c051-549b-4a53-9a61-8c88bac86f85 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:44.827507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b83f1-daef-475f-a405-bbf329e994c3 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:44.848666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08087385-24aa-4519-9fe9-32ca03dbd3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:44.849296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08087385-24aa-4519-9fe9-32ca03dbd3e7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.849592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08087385-24aa-4519-9fe9-32ca03dbd3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.849864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08087385-24aa-4519-9fe9-32ca03dbd3e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.850950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08087385-24aa-4519-9fe9-32ca03dbd3e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:44.852377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08087385-24aa-4519-9fe9-32ca03dbd3e7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:44.854260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 109/215] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:44.860799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1e24-2da1-40ef-b576-01cccff8dc28 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.860799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1e24-2da1-40ef-b576-01cccff8dc28 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.860799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d1e24-2da1-40ef-b576-01cccff8dc28 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.860799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 110/216] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:44.862154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1e24-2da1-40ef-b576-01cccff8dc28 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:44.871760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d98aa6-dafb-40b6-ac7a-15bd57faf0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:44.872409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d98aa6-dafb-40b6-ac7a-15bd57faf0e5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.872690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d98aa6-dafb-40b6-ac7a-15bd57faf0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.872966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d98aa6-dafb-40b6-ac7a-15bd57faf0e5 None admin] PATH_INFO: `/v3/users/fefbf23460db47ff8f9e64090e2ba469` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.890715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d98aa6-dafb-40b6-ac7a-15bd57faf0e5 None admin] RBAC: Authorizing `identity:get_user(user_id=fefbf23460db47ff8f9e64090e2ba469)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:44.892199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d98aa6-dafb-40b6-ac7a-15bd57faf0e5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:44.898439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 107/217] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:44 2022] GET /identity/v3/users/fefbf23460db47ff8f9e64090e2ba469 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:44.898439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:44.898439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:44.898439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:44.914055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:44.915245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:44.938820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:44.979590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d1e24-2da1-40ef-b576-01cccff8dc28 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:44.987118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:44.991594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 108/218] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:44.992279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.029222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01490369-1d66-4996-a331-bf53b58eb1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.029864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01490369-1d66-4996-a331-bf53b58eb1ed None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.030145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01490369-1d66-4996-a331-bf53b58eb1ed None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.030419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01490369-1d66-4996-a331-bf53b58eb1ed None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.031072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01490369-1d66-4996-a331-bf53b58eb1ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:45.032524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01490369-1d66-4996-a331-bf53b58eb1ed None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:45.061598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 111/219] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:44 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:45.061598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020284a-9c11-4a32-9025-ef5e40240abd None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.061598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020284a-9c11-4a32-9025-ef5e40240abd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.061598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c020284a-9c11-4a32-9025-ef5e40240abd None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.061598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 112/220] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:45.065259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.065491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.065794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.085832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:45.087151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:45.118336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259da82-2c7e-4260-a19a-2d8f94ff59df None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.160348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.161396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.161877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.162122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 113/221] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:45.162530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.192227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:45.240091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:45.241708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:45.262167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.382795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 109/222] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:45.389596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.394998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:45.407920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32ecb2-1458-44b3-a57e-12e578df0131 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.454764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.455414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.455697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.455973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.469117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:45.471850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:45.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:45.474890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:45.475655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 114/223] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:45.477698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.539080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6274c86-9039-4ad2-9c5b-5f4e5f866789 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.582297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.583044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.583328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.583667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.584826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:45.586547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:45.588573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 110/224] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:45.589983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.610788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d4843-927e-4ff4-bf59-0f479d40225e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.648331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.648896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.649177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.649451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.660983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:45.665926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:45.667307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:45.668068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 115/225] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:45.668697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.687489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:45.718759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e7566-bf86-4b46-8323-307245e6d90c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.750876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:45.765399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.766930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.767692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.768165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.769293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:45.770205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:45.771828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:45.785807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 111/226] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:45.787021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.787126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e2714c-f15e-47bb-bc46-4da3764bc585 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.828668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.828897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.829150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.829429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.830250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=fefbf23460db47ff8f9e64090e2ba469) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:45.830561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:45.832030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:45.838298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 116/227] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/role_assignments?user.id=fefbf23460db47ff8f9e64090e2ba469&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:45.839135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:45.904731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead4aed-873f-4645-ad3b-0ecbce95ebbd None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.932221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:45.946073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:45.947061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:45.947594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:45.948663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:45.949542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:45.950095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:45.951906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:45.961841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 112/228] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:45.962232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffffc26-fc7f-430f-9c44-51e94ff21b3d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:45.967770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:46.007445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:46.008748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:46.009291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:46.009840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] PATH_INFO: `/v3/users/535f7ae859e047bb869814b1ed485184` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:46.012462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] RBAC: Authorizing `identity:get_user(user_id=535f7ae859e047bb869814b1ed485184)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:46.014347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:46.016440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 117/229] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/users/535f7ae859e047bb869814b1ed485184 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:46.021420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:46.026243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:46.089747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbc6884-f454-49af-8254-3a7e8e4e89a7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:46.133810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:46.134792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:46.135270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:46.135769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] PATH_INFO: `/v3/users/535f7ae859e047bb869814b1ed485184` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:46.137255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] RBAC: Authorizing `identity:get_user(user_id=535f7ae859e047bb869814b1ed485184)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:46.140298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:46.142659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 113/230] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:45 2022] GET /identity/v3/users/535f7ae859e047bb869814b1ed485184 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:46.149058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:46.171134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3082501-8535-47c6-a5be-f3cd4099b8bd None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:46.230463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:46.231503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:46.232025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:46.232490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:46.233935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:46.235692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:46.238794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 118/231] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:46 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:46.243290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:46.269656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e0ec7-006a-4cdd-95f3-cfdcaa81b6fa None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:46.273596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:46.289433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:46.320805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:46.321464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:46.321773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:46.322047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:46.329815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:46.331313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:46.333099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 114/232] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:46 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:46.339055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:46.409004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22115644-4a6c-4be1-9524-1bd43f27e530 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:46.453152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4160c-f5ef-4a50-9721-e1eb599717f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:46.453839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4160c-f5ef-4a50-9721-e1eb599717f0 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:46.454121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4160c-f5ef-4a50-9721-e1eb599717f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:46.454396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4160c-f5ef-4a50-9721-e1eb599717f0 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/535f7ae859e047bb869814b1ed485184/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:46.456429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4160c-f5ef-4a50-9721-e1eb599717f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3d1f0afb5114fc887e7a1565d3707b0, user_id=535f7ae859e047bb869814b1ed485184, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:46.458033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4160c-f5ef-4a50-9721-e1eb599717f0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:46.491157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 119/233] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:46 2022] PUT /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/535f7ae859e047bb869814b1ed485184/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:46.513078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c06246-36d2-437b-88fc-56a4eca7fe8e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:46.567667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b2677-cc85-497f-a55f-202ad2a992a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:46.568436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b2677-cc85-497f-a55f-202ad2a992a1 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:46.568723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b2677-cc85-497f-a55f-202ad2a992a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:46.568997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b2677-cc85-497f-a55f-202ad2a992a1 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/535f7ae859e047bb869814b1ed485184/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:46.571341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b2677-cc85-497f-a55f-202ad2a992a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3d1f0afb5114fc887e7a1565d3707b0, user_id=535f7ae859e047bb869814b1ed485184, role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:46.572863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b2677-cc85-497f-a55f-202ad2a992a1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:46.598810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 115/234] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:46 2022] PUT /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/535f7ae859e047bb869814b1ed485184/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:47.482458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786979c3-6403-4405-9794-9dc7112d53b2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:47.482956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786979c3-6403-4405-9794-9dc7112d53b2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:47.483029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786979c3-6403-4405-9794-9dc7112d53b2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:47.484629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 120/235] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:47.489611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3229e-e581-494f-bedc-5a9660c4b20a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:47.489874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3229e-e581-494f-bedc-5a9660c4b20a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:47.490147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3229e-e581-494f-bedc-5a9660c4b20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:47.518563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1ba3229e-e581-494f-bedc-5a9660c4b20a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:47.519772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1ba3229e-e581-494f-bedc-5a9660c4b20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:47.577298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:47.617351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 116/236] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:47.687795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1277f02-d6b8-44c0-9bc4-5c787799473a None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:47.688408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1277f02-d6b8-44c0-9bc4-5c787799473a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:47.688876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1277f02-d6b8-44c0-9bc4-5c787799473a None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:47.690467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 121/237] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:47.695987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:47.696498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:47.696966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:47.723307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:47.724812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:47.827520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 117/238] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:47.982863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1277f02-d6b8-44c0-9bc4-5c787799473a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:48.079465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:48.123735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1277f02-d6b8-44c0-9bc4-5c787799473a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:48.174944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:48.176566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:48.177210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.177487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.177778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.197923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:48.207185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:48.208024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:48.208787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:48.212005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 122/239] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:48.218797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:48.372845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c501b-16ab-4001-b46d-9f8f830619ae None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:48.377437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:48.407377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efd66ea-651d-4b2e-b26a-244d92cc0816 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:48.420835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243bc6e-3f98-4f45-8f21-affbf3cd6ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:48.421737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243bc6e-3f98-4f45-8f21-affbf3cd6ed7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.422159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243bc6e-3f98-4f45-8f21-affbf3cd6ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.422605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243bc6e-3f98-4f45-8f21-affbf3cd6ed7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.423858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243bc6e-3f98-4f45-8f21-affbf3cd6ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:48.425443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243bc6e-3f98-4f45-8f21-affbf3cd6ed7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:48.427527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 118/240] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:48.428839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062188-2f8c-4951-a9e5-2ec2ecb353fd None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.429334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062188-2f8c-4951-a9e5-2ec2ecb353fd None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.429827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe062188-2f8c-4951-a9e5-2ec2ecb353fd None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.431330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 119/241] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:48.437538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062188-2f8c-4951-a9e5-2ec2ecb353fd None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:48.445620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:48.453849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:48.454785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.455240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.456253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.457538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:48.458053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:48.460163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485b9db-a4a5-458b-b540-5b047004e641 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:48.472528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 123/242] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:48.475191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.475656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.476414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.500710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:48.503645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:48.565170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe062188-2f8c-4951-a9e5-2ec2ecb353fd None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:48.599619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 124/243] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:48.600815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:48.609530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:48.610290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.610679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.611043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.611814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:48.613431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:48.675033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 120/244] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:48 2022] POST /identity/v3/users => generated 293 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:48.722498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2bfca-94b9-4619-9c6c-427966acb6a8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:48.751889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:48.765847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:48.766839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.767270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.767743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] PATH_INFO: `/v3/users/bcd6f365a76f4009a25c6deab05dde49` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:48.784795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] RBAC: Authorizing `identity:get_user(user_id=bcd6f365a76f4009a25c6deab05dde49)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:48.786563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:48.788932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 125/245] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:48 2022] GET /identity/v3/users/bcd6f365a76f4009a25c6deab05dde49 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:48.795036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:48.847184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:48.926726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dfedf2-76cf-4b32-886d-383c57cf2bd6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:48.963537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedd215-d61d-45cd-9be0-d3d3ed5ae61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:48.964210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedd215-d61d-45cd-9be0-d3d3ed5ae61c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:48.964495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedd215-d61d-45cd-9be0-d3d3ed5ae61c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:48.964768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedd215-d61d-45cd-9be0-d3d3ed5ae61c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.003895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.048935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedd215-d61d-45cd-9be0-d3d3ed5ae61c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:49.050764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedd215-d61d-45cd-9be0-d3d3ed5ae61c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:49.083676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:49.141239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443dcd1c-ebc9-4b8b-b0a1-1791dfb5b097 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:49.176222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 121/246] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:49.177633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddddc49-edbe-4373-ac39-c49b5b919c34 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.177887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddddc49-edbe-4373-ac39-c49b5b919c34 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.178135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddddc49-edbe-4373-ac39-c49b5b919c34 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.179781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 122/247] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:49.180895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fbead-fd8a-447b-99eb-4447549a9718 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.181173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fbead-fd8a-447b-99eb-4447549a9718 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.181443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fbead-fd8a-447b-99eb-4447549a9718 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.183437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 123/248] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:49.186287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d40ee6-f38c-4ddf-8f05-66a2e86d08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:49.186973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d40ee6-f38c-4ddf-8f05-66a2e86d08ed None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.187277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d40ee6-f38c-4ddf-8f05-66a2e86d08ed None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.187820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d40ee6-f38c-4ddf-8f05-66a2e86d08ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.189224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.189724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.190323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.200406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d40ee6-f38c-4ddf-8f05-66a2e86d08ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-27d40ee6-f38c-4ddf-8f05-66a2e86d08ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:49.205450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:49.206419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:49.207194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 126/249] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:49.208311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.208469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.208703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.214062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:49.215327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:49.227887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:49.230089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:49.279028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:49.292994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 124/250] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:49.294034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.330189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 127/251] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:49.332046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.428267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:49.503834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89293ac4-0e33-4b2b-9fcc-7fcd6859e41c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:49.536917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65705948-e7dc-4499-8d6f-e22390b74349 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:49.572003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:49.572621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.572966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.573775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.574443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:49.574744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:49.576127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:49.585000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 128/252] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:49.592036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.598286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:49.599047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.599334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.599610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.600795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:49.602284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:49.604227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 125/253] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:49.610407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.629955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:49.695839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9932f1-09d1-4156-8912-bf42e14d9625 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:49.731364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:49.732012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.732296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.732575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] PATH_INFO: `/v3/users/fefbf23460db47ff8f9e64090e2ba469` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.733756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] RBAC: Authorizing `identity:get_user(user_id=fefbf23460db47ff8f9e64090e2ba469)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:49.735234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:49.737205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 129/254] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/users/fefbf23460db47ff8f9e64090e2ba469 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:49.742497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.800831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896ad4d-3017-4ea4-8aec-7176121f60e7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:49.829833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:49.843908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:49.844534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.844823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.845100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.845944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] RBAC: Adding query filter params (role.id=14254fae7ef74b73870f800cbc43540a, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=bcd6f365a76f4009a25c6deab05dde49) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:49.846259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:49.847714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:49.855508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 126/255] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/role_assignments?user.id=bcd6f365a76f4009a25c6deab05dde49&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=14254fae7ef74b73870f800cbc43540a => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:49.856934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:49.905873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67e557-95d8-450f-8017-835f5c894b68 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:49.910615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:49.990598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:49.991832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:49.992309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:49.993193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:49.994597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:49.996475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:50.004695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 130/256] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:50.006136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.029773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86323bc0-338b-46b0-8d08-ac236d4ce670 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.101607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.102676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.103151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.103884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.127198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:50.132585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:50.133418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:50.134633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 127/257] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:32:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:50.135560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.199771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:50.211030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea4b6b9-cd12-45c0-9d3d-1d380509c19a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.285422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.286089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.286363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.286632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.293858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e55c6-666c-41e2-85e6-f63734eb576a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.309853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:50.315533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:50.316380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 131/258] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:50.317488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.361855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.362534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.362814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.363091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/fefbf23460db47ff8f9e64090e2ba469/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.365817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] RBAC: Authorizing `identity:create_grant(project_id=b3d1f0afb5114fc887e7a1565d3707b0, user_id=fefbf23460db47ff8f9e64090e2ba469, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:50.367285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:50.374021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:50.394501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 128/259] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:50 2022] PUT /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/fefbf23460db47ff8f9e64090e2ba469/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:50.395339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.509659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335bccf-036b-43ce-a6cb-5698c374f26d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.579715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.580342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.580604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.580876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.581553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:50.581877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:50.583277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:50.597958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 132/260] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:50.601377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c446f7-bb8a-46f2-a7d4-1f7155f4a03a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.604327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.627096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:50.679864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.680508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.680788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.681060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.681761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:50.682095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:50.684472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:50.711789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 129/261] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:50.722790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.732381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:50.740718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a63e09-dd2f-4bf1-829e-f4c1761dd6ca None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.790174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.790969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.791348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.792237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] PATH_INFO: `/v3/users/535f7ae859e047bb869814b1ed485184` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.794077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] RBAC: Authorizing `identity:get_user(user_id=535f7ae859e047bb869814b1ed485184)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:50.796196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:50.799065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 133/262] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/users/535f7ae859e047bb869814b1ed485184 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:50.804174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.850507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5284dea-6691-4aa7-93e1-9751a87504e7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.895619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:50.896507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:50.896932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:50.897372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] PATH_INFO: `/v3/users/535f7ae859e047bb869814b1ed485184` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:50.900487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] RBAC: Authorizing `identity:get_user(user_id=535f7ae859e047bb869814b1ed485184)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:50.902333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:50.904409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 130/263] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/users/535f7ae859e047bb869814b1ed485184 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:50.909538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:50.972921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae1c89-79ef-4952-83d9-9390ad88ee78 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:50.991944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:51.021136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c5053-13fd-413c-a66b-5a30471a597b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:51.022485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c5053-13fd-413c-a66b-5a30471a597b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.022762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c5053-13fd-413c-a66b-5a30471a597b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.023031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c5053-13fd-413c-a66b-5a30471a597b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.090355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:51.108964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745f176-8cef-4e8f-8466-2fd8aaf25f09 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:51.163058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c5053-13fd-413c-a66b-5a30471a597b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:51.164549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c5053-13fd-413c-a66b-5a30471a597b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:51.168945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ed82a-23fb-4dbf-8c17-235ad9c05ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:51.169578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ed82a-23fb-4dbf-8c17-235ad9c05ac3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.169864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ed82a-23fb-4dbf-8c17-235ad9c05ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.170134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ed82a-23fb-4dbf-8c17-235ad9c05ac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.301080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ed82a-23fb-4dbf-8c17-235ad9c05ac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:51.302963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ed82a-23fb-4dbf-8c17-235ad9c05ac3 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:51.348064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 134/264] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:32:51.350718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ee02-0be2-4a14-88b2-59564ca81d26 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.351886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ee02-0be2-4a14-88b2-59564ca81d26 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.352356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6ee02-0be2-4a14-88b2-59564ca81d26 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.356238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 135/265] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:51.357289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ee02-0be2-4a14-88b2-59564ca81d26 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:51.376172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:51.462898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:51.510981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 131/266] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:51.513520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.514134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.514696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.535085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:51.536598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:51.563365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6ee02-0be2-4a14-88b2-59564ca81d26 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:51.617681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 132/267] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:51.619460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:51.662345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1144157-1784-46fe-bc61-822c77aa0953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:51.662987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144157-1784-46fe-bc61-822c77aa0953 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.663199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144157-1784-46fe-bc61-822c77aa0953 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.663478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1144157-1784-46fe-bc61-822c77aa0953 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.664655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144157-1784-46fe-bc61-822c77aa0953 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:51.671516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1144157-1784-46fe-bc61-822c77aa0953 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:51.671516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 136/268] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:51 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:51.671516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a617b3-7562-4908-b650-a3786bd92f61 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.671516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a617b3-7562-4908-b650-a3786bd92f61 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.671516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a617b3-7562-4908-b650-a3786bd92f61 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.676387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 137/269] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:51.677136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a617b3-7562-4908-b650-a3786bd92f61 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:51.722949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:51.779971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952240c-10ff-488f-b34d-0841edfb3952 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:51.842836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df442a2-fad1-4e48-a625-dbd4bd6dcfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:51.843488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df442a2-fad1-4e48-a625-dbd4bd6dcfe8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.843766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df442a2-fad1-4e48-a625-dbd4bd6dcfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.844037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df442a2-fad1-4e48-a625-dbd4bd6dcfe8 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.845216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df442a2-fad1-4e48-a625-dbd4bd6dcfe8 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:51.846704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df442a2-fad1-4e48-a625-dbd4bd6dcfe8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:51.848351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 133/270] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:51 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:51.849462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.849766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.850039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.865743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a617b3-7562-4908-b650-a3786bd92f61 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:51.887957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:51.889174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:51.915973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:51.930340 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:51.930947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:51.931225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:51.931496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:51.932759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=535f7ae859e047bb869814b1ed485184) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:51.933071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:51.935586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:51.943596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 138/271] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:51 2022] GET /identity/v3/role_assignments?user.id=535f7ae859e047bb869814b1ed485184&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:51.944266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.032380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 134/272] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:52.033352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.127283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606381d4-e58c-4f14-98d6-bb2c6e7a428e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.151995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:52.162689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.163301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.163581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.163857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.164589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] RBAC: Adding query filter params (role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=535f7ae859e047bb869814b1ed485184) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:52.164909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:52.166422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:52.172991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 139/273] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:51 2022] GET /identity/v3/role_assignments?user.id=535f7ae859e047bb869814b1ed485184&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:52.197620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac65f3b-06f6-4cb1-8747-1836dc58f276 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.249610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30aea4-a05e-4dff-9837-eed2c8be27af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.250622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30aea4-a05e-4dff-9837-eed2c8be27af None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.251122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30aea4-a05e-4dff-9837-eed2c8be27af None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.251588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30aea4-a05e-4dff-9837-eed2c8be27af None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.264826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30aea4-a05e-4dff-9837-eed2c8be27af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-ff30aea4-a05e-4dff-9837-eed2c8be27af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:52.276963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:52.277922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:52.278670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 135/274] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:52.282705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.366555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:52.386991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a28d8e-0b60-4e9d-a12a-54cb241c21e6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.387302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a28d8e-0b60-4e9d-a12a-54cb241c21e6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.387550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a28d8e-0b60-4e9d-a12a-54cb241c21e6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.389053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 136/275] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:52.395266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.395843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.396334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.417152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ecd5e-3155-4624-a60c-7a15936fa586 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.418491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:52.420087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:52.453791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.454515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.454844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.455120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.455837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:52.456176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:52.457607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:52.467152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 140/276] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:52.472906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.521532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 137/277] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:52.525595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.601889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe721f01-5392-41f7-9bf2-cb5110b1f419 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.612733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:52.625596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:52.647715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.649116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.649592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.650041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] PATH_INFO: `/v3/users/fa6c5465aa814713ae1740c019a03bbf` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.663193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e75826-4a9e-4147-a379-bd2aee5ba41d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.670200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6c5465aa814713ae1740c019a03bbf)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:52.672658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:52.675238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 141/278] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:52 2022] GET /identity/v3/users/fa6c5465aa814713ae1740c019a03bbf => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:52.681275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.705141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.706301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.706512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.706886 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.708245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:52.710018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:52.711674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 138/279] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:52.718489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.795098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2e767-6803-471c-a8e6-a450011e4127 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.835734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637ad3-9fdf-463a-8e26-cb29fc95e248 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:52.835978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.836740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.837184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.837617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.839089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, user.id=fa6c5465aa814713ae1740c019a03bbf, scope.system=all) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:52.839544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:52.841206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:52.847238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 142/280] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 05:32:52 2022] GET /identity/v3/role_assignments?user.id=fa6c5465aa814713ae1740c019a03bbf&scope.system=all&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:52.866328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:52.876460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:52.877107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:52.877381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:52.877711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:52.878447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:52.880051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:52.893284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:52.908356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 139/281] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:32:52 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:52.945427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:53.017774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff59519c-459a-4a9c-9cfd-380a5fe50e89 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:53.056189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482d985-8751-44f8-8237-99be6660ebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:53.056844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d985-8751-44f8-8237-99be6660ebbd None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.057125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d985-8751-44f8-8237-99be6660ebbd None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.058035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482d985-8751-44f8-8237-99be6660ebbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.069993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482d985-8751-44f8-8237-99be6660ebbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7482d985-8751-44f8-8237-99be6660ebbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:53.075448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:53.076324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:53.077195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 143/282] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:53.082451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:53.166386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d4d50-1a7f-4017-a5ab-81b041608876 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.167540 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d4d50-1a7f-4017-a5ab-81b041608876 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.168029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d4d50-1a7f-4017-a5ab-81b041608876 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.169600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 144/283] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:53.176676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.177265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.177846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.200818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:53.201984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:53.220796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a602d8b-f696-45f6-9c66-dedd43726255 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:53.262148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:53.263214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.263777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.264230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.265394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] RBAC: Adding query filter params (name=reader) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:53.265902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:53.268211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:53.280311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 140/284] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:53.285904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:53.296616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 145/285] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:53.339133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:53.398408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac2ac9-3012-4cb3-a4db-d7e2a3e3fcda None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:53.435663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:53.436538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.436993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.437406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] PATH_INFO: `/v3/users/bcd6f365a76f4009a25c6deab05dde49` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.438774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] RBAC: Authorizing `identity:get_user(user_id=bcd6f365a76f4009a25c6deab05dde49)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:53.440419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:53.442553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 141/286] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:53 2022] GET /identity/v3/users/bcd6f365a76f4009a25c6deab05dde49 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:53.448301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:53.481212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:53.567843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b409f9-a2b8-481c-94b5-4f369aa8dcba None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:53.592511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:53.620729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:53.621496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.621848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.622177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.626270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:53.627974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:53.629979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 146/287] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:53 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:53.635733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:53.763491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:53.766972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd8bd8b-7f4f-4bae-a5ac-f06c8e0a5496 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:53.768773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a1c6c2-f926-4935-88ca-f5dc3c5f2be0 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:53.792356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:53.807167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:53.808231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.808636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.809037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/bcd6f365a76f4009a25c6deab05dde49/roles/14254fae7ef74b73870f800cbc43540a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.811361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] RBAC: Authorizing `identity:create_grant(project_id=b3d1f0afb5114fc887e7a1565d3707b0, user_id=bcd6f365a76f4009a25c6deab05dde49, role_id=14254fae7ef74b73870f800cbc43540a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:53.813086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:53.827932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:53.828916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:53.829583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:53.830002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:53.859056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 147/288] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:32:53 2022] PUT /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/users/bcd6f365a76f4009a25c6deab05dde49/roles/14254fae7ef74b73870f800cbc43540a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:53.859771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:53.865694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:53.866637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:53.867383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:53.872496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 142/289] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:53.874048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:53.997466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bb8e92-3365-48ac-9d0e-459f0a45139b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:54.037192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:54.037763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:54.038113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:54.038381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:54.039061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:54.039359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:54.040841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:54.054519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 148/290] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:54.058231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:54.169229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8ddb2-4793-4a5e-ba3b-484483213cb2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:54.206381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:54.207065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:54.207347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:54.207612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] PATH_INFO: `/v3/users/fefbf23460db47ff8f9e64090e2ba469` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:54.208740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] RBAC: Authorizing `identity:get_user(user_id=fefbf23460db47ff8f9e64090e2ba469)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:54.210270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:54.212180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 143/291] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:54 2022] GET /identity/v3/users/fefbf23460db47ff8f9e64090e2ba469 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:54.219789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:54.251877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:54.380420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40df7b1e-14c0-432b-922a-4adab3395f8e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:54.428666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:54.429281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:54.429572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:54.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:54.515492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:54.516958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:54.549514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:54.650226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 149/292] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:54.656391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:54.726101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:54.804108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dccc96-b8b5-4d24-8652-d5cea6cb7db6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:54.853003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe7c5b-9728-47d5-b282-4548b8da625c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:54.853920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe7c5b-9728-47d5-b282-4548b8da625c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:54.854227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe7c5b-9728-47d5-b282-4548b8da625c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:54.854497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe7c5b-9728-47d5-b282-4548b8da625c None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:54.855680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe7c5b-9728-47d5-b282-4548b8da625c None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:54.857130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe7c5b-9728-47d5-b282-4548b8da625c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:54.859777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 144/293] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:54 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:54.864567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:55.012276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29494c51-977d-4ac6-9ab0-c355036a5c11 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:55.040535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:55.062237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:55.062855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.063142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.063420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.064157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=fefbf23460db47ff8f9e64090e2ba469) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:55.064469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:55.065933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8830cdb-6856-4d1c-b3a0-bcfbab193034 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:55.072269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 150/294] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:54 2022] GET /identity/v3/role_assignments?user.id=fefbf23460db47ff8f9e64090e2ba469&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:55.289056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa743a6-5ea5-412c-95c8-3e1be5ad0d0a None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.289381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa743a6-5ea5-412c-95c8-3e1be5ad0d0a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.289672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa743a6-5ea5-412c-95c8-3e1be5ad0d0a None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.291288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 145/295] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:55.296827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5d2f-700c-4c1b-90f2-348f3deb3e5e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.297113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5d2f-700c-4c1b-90f2-348f3deb3e5e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.297381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690c5d2f-700c-4c1b-90f2-348f3deb3e5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.318051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-690c5d2f-700c-4c1b-90f2-348f3deb3e5e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:55.319259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-690c5d2f-700c-4c1b-90f2-348f3deb3e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:55.408459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 151/296] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:55.503582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6708e1-52db-4d1e-b4e6-cfb0390df7f4 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.504181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6708e1-52db-4d1e-b4e6-cfb0390df7f4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.504620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6708e1-52db-4d1e-b4e6-cfb0390df7f4 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.506145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 146/297] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:55.512391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.512679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.512952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.536811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:55.537986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:55.611335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 152/298] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:55.694455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6708e1-52db-4d1e-b4e6-cfb0390df7f4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:55.808224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6708e1-52db-4d1e-b4e6-cfb0390df7f4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:55.817451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:55.844322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18546146-4ce4-4ab8-a24a-e292b410cab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:55.845009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18546146-4ce4-4ab8-a24a-e292b410cab1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.845294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18546146-4ce4-4ab8-a24a-e292b410cab1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.845586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18546146-4ce4-4ab8-a24a-e292b410cab1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.857155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18546146-4ce4-4ab8-a24a-e292b410cab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-18546146-4ce4-4ab8-a24a-e292b410cab1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:55.862098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:55.862954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:55.863704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 147/299] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:55.870985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:55.887204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8b702-c1c5-4ce8-846f-e79459f35c30 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.887502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8b702-c1c5-4ce8-846f-e79459f35c30 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.887741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f8b702-c1c5-4ce8-846f-e79459f35c30 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.890366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 148/300] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:55.894505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:55.894789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:55.895055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:55.915593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:55.918008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:55.922277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:55.985998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea2189-f01b-44f7-a8fd-74f73feab423 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:55.989350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 149/301] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:55.994201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.025601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416e247-413d-45e8-945c-d9921072984f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.027259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416e247-413d-45e8-945c-d9921072984f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.027753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416e247-413d-45e8-945c-d9921072984f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.028323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416e247-413d-45e8-945c-d9921072984f None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.029285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416e247-413d-45e8-945c-d9921072984f None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:56.029847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416e247-413d-45e8-945c-d9921072984f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.031555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416e247-413d-45e8-945c-d9921072984f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.041357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 153/302] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:56.046451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416e247-413d-45e8-945c-d9921072984f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.131757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:56.137420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5ab0f-6bea-4ffd-b115-47d2afaad340 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.168019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416e247-413d-45e8-945c-d9921072984f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.175425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:56.176322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49630cdc-43c8-4c57-9e7d-b788cdfcacc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.176983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630cdc-43c8-4c57-9e7d-b788cdfcacc6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.177281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630cdc-43c8-4c57-9e7d-b788cdfcacc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.177509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49630cdc-43c8-4c57-9e7d-b788cdfcacc6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.189430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49630cdc-43c8-4c57-9e7d-b788cdfcacc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-49630cdc-43c8-4c57-9e7d-b788cdfcacc6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:56.194229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:56.195115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:56.195870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 150/303] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:56.198208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de36dc-82ed-4fd5-9daa-9c352e7d9dfb None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.198208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de36dc-82ed-4fd5-9daa-9c352e7d9dfb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.198208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4de36dc-82ed-4fd5-9daa-9c352e7d9dfb None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.202900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 151/304] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:56.202900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de36dc-82ed-4fd5-9daa-9c352e7d9dfb None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.213277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8381fc0-03c4-4267-a9ef-8d83f296553b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.214494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8381fc0-03c4-4267-a9ef-8d83f296553b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.214857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8381fc0-03c4-4267-a9ef-8d83f296553b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.215144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8381fc0-03c4-4267-a9ef-8d83f296553b None admin] PATH_INFO: `/v3/users/fa6c5465aa814713ae1740c019a03bbf` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.216266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8381fc0-03c4-4267-a9ef-8d83f296553b None admin] RBAC: Authorizing `identity:get_user(user_id=fa6c5465aa814713ae1740c019a03bbf)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.217957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8381fc0-03c4-4267-a9ef-8d83f296553b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.221774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 154/305] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/users/fa6c5465aa814713ae1740c019a03bbf => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:56.221774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.221774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.221774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.240367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:56.241553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:56.310555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 155/306] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:56.311524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.311866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4de36dc-82ed-4fd5-9daa-9c352e7d9dfb None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.358279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.359159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.359616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.360060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.360889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:56.361387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.362999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.372683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 152/307] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:56.373790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.397294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:56.417499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cafdc-acee-41ab-9fa5-5ceb83848556 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.419854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:56.452735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.453397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.453712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.453996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] PATH_INFO: `/v3/system/users/fa6c5465aa814713ae1740c019a03bbf/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.455595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fa6c5465aa814713ae1740c019a03bbf, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.456975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.468247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 156/308] 10.222.0.73 () {58 vars in 1334 bytes} [Wed Aug 24 05:32:56 2022] PUT /identity/v3/system/users/fa6c5465aa814713ae1740c019a03bbf/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:32:56.469108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.495721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115a6af-155b-401b-a37d-78a1762b89f2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.536786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.537891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.538349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.538882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.540301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.542781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.545199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 153/309] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:56.555824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.631377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee9780b-3916-4eb7-b613-2f0db6cf51f8 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.662259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:56.676034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201373df-1711-48de-9c0a-c6516e4aa2b8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.676590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.677233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.677515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.677817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] PATH_INFO: `/v3/users/1d6e7f7c605f46e99764d7f0ccc60a48` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.695356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6e7f7c605f46e99764d7f0ccc60a48)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.697265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.704385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 157/310] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/users/1d6e7f7c605f46e99764d7f0ccc60a48 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:56.704385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.711793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.712400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.712677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.712943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] PATH_INFO: `/v3/groups` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.713602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.715064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.734005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 154/311] 10.222.0.73 () {60 vars in 1134 bytes} [Wed Aug 24 05:32:56 2022] POST /identity/v3/groups => generated 229 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:56.735194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.742627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:56.840515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670edec-9619-46b2-b3db-16c98f4d7b79 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.880099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.880774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.881064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.881304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.893604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:56.897050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4668b7-e381-45c0-bf92-427186eebb94 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:56.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:56.899324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:56.900098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:32:56.905039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 158/312] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:56.905984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:56.949352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:56.950309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:56.950604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:56.950866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:56.951765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] RBAC: Adding query filter params (role.id=14254fae7ef74b73870f800cbc43540a, user.id=1d6e7f7c605f46e99764d7f0ccc60a48, scope.system=all) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:56.952082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:56.954455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:56.961871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 155/313] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/role_assignments?user.id=1d6e7f7c605f46e99764d7f0ccc60a48&scope.system=all&role.id=14254fae7ef74b73870f800cbc43540a => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:57.022025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:57.035979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9b811-b16a-4329-8d4e-9fe12bf352d2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:57.075014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:57.075652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:57.075932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:57.076200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:57.076883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:57.077186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:57.078744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:57.088481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 159/314] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:57.094021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:57.111572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:57.202600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f64a19-3190-4087-a0a3-bb596bb67d3d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:57.238648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:57.239333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:57.239613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:57.239939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] PATH_INFO: `/v3/users/bcd6f365a76f4009a25c6deab05dde49` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:57.241094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] RBAC: Authorizing `identity:get_user(user_id=bcd6f365a76f4009a25c6deab05dde49)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:57.242562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:57.244467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 156/315] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:57 2022] GET /identity/v3/users/bcd6f365a76f4009a25c6deab05dde49 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:57.250695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:57.363995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d87fc8-7d61-47ca-913f-c556b0b689a9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:57.399213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:57.399987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:57.400341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:57.400687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:57.421778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:57.479229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:57.481132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:57.592906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 160/316] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:32:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:32:57.598296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:57.684939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:57.704134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f02a65-2533-49df-81b0-a5ab16303885 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:57.739480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e1569a-de52-4071-be6d-56eae091dfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:57.740038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1569a-de52-4071-be6d-56eae091dfb8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:57.740314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1569a-de52-4071-be6d-56eae091dfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:57.740583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e1569a-de52-4071-be6d-56eae091dfb8 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:57.741732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1569a-de52-4071-be6d-56eae091dfb8 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:57.743173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e1569a-de52-4071-be6d-56eae091dfb8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:57.745033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 157/317] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:32:57 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:57.750708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:57.803248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:57.855527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a71a30-cc2f-4dc1-a616-5da68b7a6817 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:57.891223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:57.891885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:57.892100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:57.892372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:57.893103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] RBAC: Adding query filter params (role.id=14254fae7ef74b73870f800cbc43540a, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0, user.id=bcd6f365a76f4009a25c6deab05dde49) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:57.893414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:57.894901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93040492-27ad-41c6-8ccc-9799475d9b2a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:57.901839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 161/318] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:32:57 2022] GET /identity/v3/role_assignments?user.id=bcd6f365a76f4009a25c6deab05dde49&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=14254fae7ef74b73870f800cbc43540a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:58.489758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ffc6-f18b-4a24-ae49-4ae9ed82f053 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:58.490284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ffc6-f18b-4a24-ae49-4ae9ed82f053 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:58.490284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4ffc6-f18b-4a24-ae49-4ae9ed82f053 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:58.491669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 158/319] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:58.497510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5691ed1-c6e3-4a9e-b0f0-2a9b5c60973d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:58.497824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5691ed1-c6e3-4a9e-b0f0-2a9b5c60973d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:58.498102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5691ed1-c6e3-4a9e-b0f0-2a9b5c60973d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:58.517115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e5691ed1-c6e3-4a9e-b0f0-2a9b5c60973d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:58.518663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e5691ed1-c6e3-4a9e-b0f0-2a9b5c60973d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:58.588200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 162/320] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:58.774948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e41c81-15b0-4026-a5f2-a6a2201152bf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:58.775304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e41c81-15b0-4026-a5f2-a6a2201152bf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:58.775498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e41c81-15b0-4026-a5f2-a6a2201152bf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:58.777677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 159/321] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:32:58.782548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cca148-f499-45d7-9797-600341d62da1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:58.782837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cca148-f499-45d7-9797-600341d62da1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:58.783111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cca148-f499-45d7-9797-600341d62da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:58.801412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a0cca148-f499-45d7-9797-600341d62da1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:58.802618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a0cca148-f499-45d7-9797-600341d62da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:58.839421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:58.858413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e41c81-15b0-4026-a5f2-a6a2201152bf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:58.873736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 163/322] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:32:58.920943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10881546-5935-4eb3-90a3-aa945545f53e None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:58.921784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10881546-5935-4eb3-90a3-aa945545f53e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:58.921784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10881546-5935-4eb3-90a3-aa945545f53e None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:58.923175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 164/323] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:32:58.927815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:58.928092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:58.928359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:58.946053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:32:58.947226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:32:58.969302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e41c81-15b0-4026-a5f2-a6a2201152bf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.005432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.006093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.006372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.006643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.017657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:59.022511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 165/324] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:32:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:59.022646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:59.023819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:32:59.024866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:59.028901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 160/325] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:32:59.030571 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.132573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe38f1a-91dc-4bf7-9eae-7f580e837e8c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.134052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:59.138371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.151740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:59.172816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.173425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.173736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.174005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.174662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:59.174961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.176320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.185154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 166/326] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:59.190354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.258326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4116a66-cfad-45d5-b197-2114283d7537 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.294697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.295328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.295604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.295873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.296995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.298489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.300053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 161/327] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:59.300808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.309189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7621d17-f42e-4fa7-a403-56d2eff7aa34 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.342390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.343012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.343287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.343559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] PATH_INFO: `/v3/users/fa6c5465aa814713ae1740c019a03bbf` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.344669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=fa6c5465aa814713ae1740c019a03bbf)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.346226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.347776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 167/328] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/users/fa6c5465aa814713ae1740c019a03bbf => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:32:59.348510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.382836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:59.383428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:59.408221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa82f80d-373f-45a5-bb1c-d0d35e57ccd2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.441811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.442438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.442714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.442985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.452480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cedb6c-6a46-409a-afbc-e3cdfb410ff9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.453781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:32:59.458577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:32:59.459458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:32:59.460207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 162/329] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:32:59.460847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.487940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.488548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.488832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.489107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] PATH_INFO: `/v3/groups` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.489760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.491204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.507616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 168/330] 10.222.0.73 () {60 vars in 1134 bytes} [Wed Aug 24 05:32:59 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:32:59.508426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.580625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2b0d9-acc9-47be-8ddf-9d4ea860538c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.610306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:59.623330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe55db2-0fad-4de3-b720-82043b125d5b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.625591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.626308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.626594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.626873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.627621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] RBAC: Adding query filter params (role.id=e6fe411570314ffdb43c23bed35b1f50, user.id=fa6c5465aa814713ae1740c019a03bbf, scope.system=all) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:59.627934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.629414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.633112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:32:59.645214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 163/331] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/role_assignments?user.id=fa6c5465aa814713ae1740c019a03bbf&scope.system=all&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:59.661660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.662297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.662583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.662832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.663511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:32:59.663807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.665216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.675497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 169/332] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:59.681446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.797132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1d16c-a07f-4fc5-9f4c-cb5fdd48521d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.835138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb20f-eb67-4581-99aa-bc805d125c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.836136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb20f-eb67-4581-99aa-bc805d125c95 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.836615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb20f-eb67-4581-99aa-bc805d125c95 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.837094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb20f-eb67-4581-99aa-bc805d125c95 None admin] PATH_INFO: `/v3/users/1d6e7f7c605f46e99764d7f0ccc60a48` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.838329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb20f-eb67-4581-99aa-bc805d125c95 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6e7f7c605f46e99764d7f0ccc60a48)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:32:59.840373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb20f-eb67-4581-99aa-bc805d125c95 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:32:59.842310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 164/333] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:32:59 2022] GET /identity/v3/users/1d6e7f7c605f46e99764d7f0ccc60a48 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:32:59.847921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:32:59.960869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9b5c3-5af7-410d-9b51-c44a5d9ae8a2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:32:59.996445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0797fba-24bb-4b9a-847e-6bc3bad366a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:32:59.997097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0797fba-24bb-4b9a-847e-6bc3bad366a3 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:32:59.997375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0797fba-24bb-4b9a-847e-6bc3bad366a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:32:59.998692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0797fba-24bb-4b9a-847e-6bc3bad366a3 None admin] PATH_INFO: `/v3/system/users/1d6e7f7c605f46e99764d7f0ccc60a48/roles/14254fae7ef74b73870f800cbc43540a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:32:59.999604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0797fba-24bb-4b9a-847e-6bc3bad366a3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1d6e7f7c605f46e99764d7f0ccc60a48, role_id=14254fae7ef74b73870f800cbc43540a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:00.000971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0797fba-24bb-4b9a-847e-6bc3bad366a3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:00.011295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 170/334] 10.222.0.73 () {58 vars in 1334 bytes} [Wed Aug 24 05:32:59 2022] PUT /identity/v3/system/users/1d6e7f7c605f46e99764d7f0ccc60a48/roles/14254fae7ef74b73870f800cbc43540a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:33:01.714303 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936eb3c-469d-4df1-a466-1ec2c1069e14 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:01.714911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936eb3c-469d-4df1-a466-1ec2c1069e14 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:01.714911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a936eb3c-469d-4df1-a466-1ec2c1069e14 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:01.716483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 165/335] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:01.725166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdc6fa-01a8-4425-b08d-139f811a59fa None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:01.725651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdc6fa-01a8-4425-b08d-139f811a59fa None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:01.725859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffdc6fa-01a8-4425-b08d-139f811a59fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:01.747877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2ffdc6fa-01a8-4425-b08d-139f811a59fa None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:01.749076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2ffdc6fa-01a8-4425-b08d-139f811a59fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:01.767109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:01.834324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 171/336] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:02.154543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ff477-cf86-43ea-bd49-96606b73de54 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.155389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ff477-cf86-43ea-bd49-96606b73de54 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.155880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ff477-cf86-43ea-bd49-96606b73de54 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.157665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 166/337] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:02.175855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.176441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.176930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.183108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a8060-024b-4c18-9366-c7b0ac0b75ce None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.183396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a8060-024b-4c18-9366-c7b0ac0b75ce None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.183678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71a8060-024b-4c18-9366-c7b0ac0b75ce None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.185248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 167/338] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:02.191809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ae184-d3e5-4067-ac6e-a662ab8c2f35 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.192092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ae184-d3e5-4067-ac6e-a662ab8c2f35 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.192366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ae184-d3e5-4067-ac6e-a662ab8c2f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.203647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:02.204850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:02.226383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ca2ae184-d3e5-4067-ac6e-a662ab8c2f35 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:02.227584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ca2ae184-d3e5-4067-ac6e-a662ab8c2f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:02.345359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 172/339] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:02.352581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:02.366117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 168/340] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:02.396049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:02.508667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:02.526306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba480ec-413d-4a68-b7e6-f8d859726979 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.526913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba480ec-413d-4a68-b7e6-f8d859726979 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.527546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba480ec-413d-4a68-b7e6-f8d859726979 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.529527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 169/341] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:02.534799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.535080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.535355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.559396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cba1fd-3e14-4484-b8f7-774d128bd7ad None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:02.574729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:02.575968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:02.605047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:02.606181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.606784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.607273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.626014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:02.631487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:02.632326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:02.633087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:02.642165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 173/342] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:02.648231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:02.712240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 170/343] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:02.781626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685bd772-dc62-45d9-bbef-59b2e59e9d37 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:02.829592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:02.830614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.831100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.831590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:02.832453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:02.832951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:02.834545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:02.840965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:02.842453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:02.845378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 174/344] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:02.851844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:02.952668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922906f-9245-4ece-98be-3e92f3a6c228 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:02.975705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdde82-8a85-4031-8cb8-eb74d5fa46bd None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:02.977811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:02.992124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:02.992854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:02.993227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:02.993589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.006084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:03.012833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:03.013731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:03.014514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 171/345] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:03.015175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.041008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.042033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.042567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.043022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.135845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2e76b-ebda-4608-9cbb-8ac1be2c47bf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.136427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:03.139641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:03.142302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:03.195194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.195831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.196113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.196389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.211368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:03.218480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:03.219385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:03.220247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:03.222445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 172/346] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:03.223791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.295244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:03.320170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 175/347] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:33:03.321333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.367683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df83f0-09cd-443c-9b89-510df7ab9b79 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.408363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.409245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.409730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.410150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.410987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:03.411503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:03.413143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:03.423859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 173/348] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:03.424981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.445347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:03.449088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87af351b-856c-4d45-af15-de2e56e206a3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.489557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.490248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.490523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.490794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.504598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:03.511286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:03.512189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:03.512951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 176/349] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:03.513634 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.550451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49aff5-afb9-48d9-831c-26d9382d393f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.553206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:03.605216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.606547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.607043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.607534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.608634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:03.609263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:03.611451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:03.622873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 174/350] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:03.624849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.675275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02496dac-1355-4719-a53e-8422109ae98c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.720497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.721137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.721417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.721723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.722861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:03.724312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:03.729648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 177/351] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:03.731495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:03.761068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526df93-4c0e-44b2-b3a7-6ee0628a32e4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.771166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:03.811308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e569fa-c3e5-4c47-b2de-af516ba0ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.811865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e569fa-c3e5-4c47-b2de-af516ba0ca99 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.812148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e569fa-c3e5-4c47-b2de-af516ba0ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.812419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e569fa-c3e5-4c47-b2de-af516ba0ca99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.857059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:03.877360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049c5dd-dc8e-4260-b56d-27e58ff8a995 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:03.903798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e569fa-c3e5-4c47-b2de-af516ba0ca99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:03.905271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e569fa-c3e5-4c47-b2de-af516ba0ca99 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:03.925107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:03.925875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:03.926241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:03.926647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:03.927300 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:03.927600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:03.928982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:03.943454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 178/352] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:03.943454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.052558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 175/353] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:04.054375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f91a0-f7f9-4d28-a314-4cfa1b2e8f2e None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.054956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f91a0-f7f9-4d28-a314-4cfa1b2e8f2e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.055404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f91a0-f7f9-4d28-a314-4cfa1b2e8f2e None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.056830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 176/354] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:04.058409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f91a0-f7f9-4d28-a314-4cfa1b2e8f2e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.066224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.075306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fba5f4-c4d2-45df-a7c2-8babfe734a44 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.120931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.120931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.121229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.121477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.148043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.197851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f91a0-f7f9-4d28-a314-4cfa1b2e8f2e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.213029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.215034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.240874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.241604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.241867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.242141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.250784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.251939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.254182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 177/355] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.255586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.339745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.355575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 179/356] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:04.356607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.389265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566c441-acb8-4779-ba61-8ad7f6a02d90 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.422336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.429786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6be11c-0c07-4646-9e6a-97b17322b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.431239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6be11c-0c07-4646-9e6a-97b17322b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.431239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6be11c-0c07-4646-9e6a-97b17322b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.431428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6be11c-0c07-4646-9e6a-97b17322b3e0 None admin] PATH_INFO: `/v3/users/1d6e7f7c605f46e99764d7f0ccc60a48` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.433178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6be11c-0c07-4646-9e6a-97b17322b3e0 None admin] RBAC: Authorizing `identity:get_user(user_id=1d6e7f7c605f46e99764d7f0ccc60a48)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.434669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6be11c-0c07-4646-9e6a-97b17322b3e0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.436447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 178/357] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/users/1d6e7f7c605f46e99764d7f0ccc60a48 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.437645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f669e-a93d-4c01-bcc9-94a5b08a2898 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.437754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f669e-a93d-4c01-bcc9-94a5b08a2898 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.438013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f669e-a93d-4c01-bcc9-94a5b08a2898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.458575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-339f669e-a93d-4c01-bcc9-94a5b08a2898 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:04.459788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-339f669e-a93d-4c01-bcc9-94a5b08a2898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:04.481703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4bf8e-c645-4ff1-9d0c-dc28aa8e0655 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.521700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.522344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.522621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.522890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.524916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.527075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.528702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 180/358] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.529459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.544423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 179/359] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:04.546197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34124cf6-f0f5-4943-99db-6d60aa6fbc5e None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.546703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34124cf6-f0f5-4943-99db-6d60aa6fbc5e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.546979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34124cf6-f0f5-4943-99db-6d60aa6fbc5e None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.548362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 180/360] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:04.548982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34124cf6-f0f5-4943-99db-6d60aa6fbc5e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.601148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.664573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f84291-1164-4217-8faa-d6d3caf273ce None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.685736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34124cf6-f0f5-4943-99db-6d60aa6fbc5e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.706775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.707380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.707667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.707942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.708672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:04.708982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.710464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.717880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 181/361] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.718995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.729477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.730696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.731199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.732550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.735295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.735818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.738017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.740251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 181/362] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.741375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.862282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af94411-be08-4223-96db-4bca47058e76 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.881499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077e017-020c-4063-893b-129c47dfa31e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:04.895991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:04.911106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.911727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.912003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.912268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.913002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] RBAC: Adding query filter params (role.id=14254fae7ef74b73870f800cbc43540a, user.id=1d6e7f7c605f46e99764d7f0ccc60a48, scope.system=all) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:04.915311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.915533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489616a-433e-4551-9565-bdcf9379c30a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.931817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:04.932756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.933038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.933304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.933556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 182/363] 10.222.0.73 () {56 vars in 1308 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/role_assignments?user.id=1d6e7f7c605f46e99764d7f0ccc60a48&scope.system=all&role.id=14254fae7ef74b73870f800cbc43540a => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.934497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:04.934852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:04.935132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:04.935451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:04.936140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:04.938108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 182/364] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:04.938857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:04.959537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:04.960783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:05.034978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:05.062456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 183/365] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:05.063546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.091847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dccf35-32ea-4808-956f-adfed3c927e4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.161603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.162685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.162959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.163230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.164571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:05.167162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:05.170376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 183/366] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:04 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:05.171478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.186628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913641a0-e729-4278-a67a-df867fb6ad8c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.200743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:05.232210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.232907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.233185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.233454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.255558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:05.259720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:05.261177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:05.262011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 184/367] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:05.262655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.307735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83003d81-dac0-48c8-b4dc-e209cfd73275 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.369879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:05.378075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.378690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.378987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.379617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.380912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:05.381244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:05.383081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:05.396211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 184/368] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:05.396211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.440389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824970a-46f7-403a-84a8-5011c86c3406 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.490099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.491107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.491593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.492070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.493056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:05.493647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:05.497009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:05.509128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 185/369] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:05.511304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.576150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:05.619875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d742b-2950-4f08-92fe-9e6b4e1777cf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.672333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.672954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.673228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.673493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.675451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227d9560-9a26-4447-8801-2d977e93edd0 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.676637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:05.676944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:05.678382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:05.699352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 185/370] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:05.704951 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.749681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.750838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.751267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.751872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.771372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:05.778171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:05.779247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:05.780211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:05.783112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 186/371] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:05.790528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:05.797616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:05.847377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24961c1-2629-4785-8306-daf4a2d99656 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.886445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.887067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.887479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.887892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.918527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:05.927105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaf864-b3aa-4561-ae04-6a3c9561a68e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:05.966605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:05.967385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:05.967872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:05.968354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:05.968842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:05.969013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:05.969905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:05.970427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:05.971981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:05.985674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 187/372] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:05.987556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.036850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:06.086537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 186/373] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:33:06.094726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.102820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234105f-f994-4b9b-8bef-689546baa571 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.140078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.140984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.141410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:06.141874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:06.170238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:06.217843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:06.219553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:06.239473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9a989-788d-41cf-875e-c9b3e1a6c831 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.296904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.297479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.297871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:06.298233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:06.301020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:06.302765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:06.304846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 187/374] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:06 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:06.310852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.337380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 188/375] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:33:06.342513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.349238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:06.430125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:06.491079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5b7554-6209-4cce-b4c1-3b2cdcce9481 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.511868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d67a03-22f0-42e0-a47f-459944e3eb2e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.530463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.531474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.531965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:06.532510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] PATH_INFO: `/v3/projects/136c4807ccbf43c49f1d29adcff44403` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:06.534110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] RBAC: Authorizing `identity:get_project(project_id=136c4807ccbf43c49f1d29adcff44403)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:06.535887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:06.541846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 189/376] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:06 2022] GET /identity/v3/projects/136c4807ccbf43c49f1d29adcff44403 => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:06.544909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.564114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.566621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.567136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:06.567629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:06.570174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:06.573200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:06.577577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 188/377] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:06 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:06.584624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.734737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bdf1fe-d4f3-43f8-bcfe-82744cae583a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.737432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e396b-380e-428c-a3fb-9db1fd424653 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.742793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:06.788505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:06.795928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.796551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.796829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:06.797101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:06.798017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:06.798324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:06.800019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96bde2-b0f0-4ce4-ac13-b22b5123f3dd None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:06.801110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.801892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.802208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:06.802548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] PATH_INFO: `/v3/groups/9edf96c20b3841dab30f060739a836ca` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:06.809952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] RBAC: Authorizing `identity:get_group(group_id=9edf96c20b3841dab30f060739a836ca)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:06.811443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:06.813537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 190/378] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:06 2022] GET /identity/v3/groups/9edf96c20b3841dab30f060739a836ca => generated 229 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:06.817880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 189/379] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:06 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:06.820085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:06.960681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01abb0c0-8d4f-4125-b328-54f975b41cd4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:06.999136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:06.999674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:06.999949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:07.000280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:07.001015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] RBAC: Adding query filter params (group.id=9edf96c20b3841dab30f060739a836ca, role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=136c4807ccbf43c49f1d29adcff44403) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:07.001323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:07.002792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba19e71-0687-4727-9b62-d1107225b2e4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:07.008654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 191/380] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:06 2022] GET /identity/v3/role_assignments?group.id=9edf96c20b3841dab30f060739a836ca&scope.project.id=136c4807ccbf43c49f1d29adcff44403&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:07.452784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6486e-6d3f-4a43-98bb-ca924abbd039 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:07.453325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6486e-6d3f-4a43-98bb-ca924abbd039 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:07.453632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6486e-6d3f-4a43-98bb-ca924abbd039 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:07.455471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 190/381] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:07.460548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fdbd5-16ba-48a3-9dba-abbef36c53ab None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:07.460837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fdbd5-16ba-48a3-9dba-abbef36c53ab None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:07.461111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fdbd5-16ba-48a3-9dba-abbef36c53ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:07.484560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1e7fdbd5-16ba-48a3-9dba-abbef36c53ab None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:07.485904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1e7fdbd5-16ba-48a3-9dba-abbef36c53ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:07.543281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:07.566269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 192/382] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:07.879094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898fe47-7693-4f89-ac47-548a2b111910 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:07.879502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898fe47-7693-4f89-ac47-548a2b111910 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:07.879702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898fe47-7693-4f89-ac47-548a2b111910 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:07.881302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 191/383] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:07.887478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1ced-09fd-42da-8678-7c3a1f7d0d09 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:07.887761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1ced-09fd-42da-8678-7c3a1f7d0d09 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:07.888034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1ced-09fd-42da-8678-7c3a1f7d0d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:07.910622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f93e1ced-09fd-42da-8678-7c3a1f7d0d09 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:07.911802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f93e1ced-09fd-42da-8678-7c3a1f7d0d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:07.999914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 193/384] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:08.020150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898fe47-7693-4f89-ac47-548a2b111910 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.110086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dddb9-c1d4-4c77-937d-6f7337948873 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.110418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dddb9-c1d4-4c77-937d-6f7337948873 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.110623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44dddb9-c1d4-4c77-937d-6f7337948873 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.112109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 194/385] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:33:08.122757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.123034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.123287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.149581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:08.157950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:08.159598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898fe47-7693-4f89-ac47-548a2b111910 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:08.204649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:08.205281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.205557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.205848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.230845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:08.238192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:08.239083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:08.240263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 192/386] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:08.246472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.289931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:08.303557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 195/387] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:08.337780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.405649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e61f31-e13e-4f60-95f1-40035d20559e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:08.431025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:08.444872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:08.445506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.445815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.446091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.446776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:08.447085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:08.448710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:08.458238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 193/388] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:08.473181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.528199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ec894-37e0-47e9-b483-6637aa8c8a3a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:08.593016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:08.593688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.593971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.594243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.605165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:08.610106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:08.610969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:08.612117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 196/389] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:08.614776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:08.617888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.661475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fda801-4d18-4dd0-b8c7-ed1b081f5feb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:08.707718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:08.708772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.709285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.709801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.711275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:08.713103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:08.715907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 194/390] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:08.716931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.755869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a3401-d7a5-4b76-9af7-7def69fa9aab None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:08.757930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:08.797326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:08.798425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.798972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.799567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.800478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] RBAC: Adding query filter params (name=member) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:08.801025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:08.802778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:08.812378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 197/391] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:08.813709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.850150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cb10c1-10dd-479d-af0b-a3e9d0cfd3b3 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:08.894839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:08.895944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:08.896497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:08.896969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:08.910389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:08.916696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:08.918367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:08.920399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:08.922640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 195/392] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:08.923469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:08.937045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:08.972647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67334dd9-bd76-478b-b2b9-003d9d610565 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.011335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.012962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.013437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.013917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.015285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.017012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.019350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 198/393] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:09.020489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.052947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9e274-08d1-49d9-80fa-0ea8c5aa3772 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.057977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:09.097369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.098057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.098337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.098619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.100328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.102114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.109433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 196/394] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:08 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:09.110292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.139902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64d2f3-0ce3-4751-8c65-9c574da45ee0 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.178752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.179680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.180270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.180734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.181721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:09.182798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.184558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.193819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 199/395] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:09.194852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.210354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:09.237877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8715a-1326-4164-af55-f6b6f87104c4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.294421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.295759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.296623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.297104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/groups/d78c75e6dc9549e9b0b744039a254e55/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.300877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] RBAC: Authorizing `identity:create_grant(project_id=68f16fe3336647ac92f5dc9a178bc2d4, group_id=d78c75e6dc9549e9b0b744039a254e55, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.303664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.321600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 197/396] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 05:33:09 2022] PUT /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/groups/d78c75e6dc9549e9b0b744039a254e55/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:09.323071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.330629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:09.341943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43d64b3-97e4-4d6a-b7fb-c5dd970881f5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.403422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2a689-17d3-4ed1-a6bc-09f232a1e52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.404390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a689-17d3-4ed1-a6bc-09f232a1e52a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.404872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a689-17d3-4ed1-a6bc-09f232a1e52a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.405275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2a689-17d3-4ed1-a6bc-09f232a1e52a None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.407408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2a689-17d3-4ed1-a6bc-09f232a1e52a None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.409979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2a689-17d3-4ed1-a6bc-09f232a1e52a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.417627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 200/397] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:09 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:09.417627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1292a4-32eb-4f9d-86cc-95c090bdefe2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.417627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1292a4-32eb-4f9d-86cc-95c090bdefe2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.417627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1292a4-32eb-4f9d-86cc-95c090bdefe2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.417627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 201/398] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:09.418485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edbc7f-d175-46c1-b28f-e88f6f7fe3dc None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.419541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edbc7f-d175-46c1-b28f-e88f6f7fe3dc None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.419541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45edbc7f-d175-46c1-b28f-e88f6f7fe3dc None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.420370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 202/399] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:09.425495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edbc7f-d175-46c1-b28f-e88f6f7fe3dc None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.476026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639847e-5ed6-46f8-98b0-d901d4609615 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.518559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77349f8a-e19a-4483-8174-14f0ab0f155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.519457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77349f8a-e19a-4483-8174-14f0ab0f155d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.520389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77349f8a-e19a-4483-8174-14f0ab0f155d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.520810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77349f8a-e19a-4483-8174-14f0ab0f155d None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.522863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77349f8a-e19a-4483-8174-14f0ab0f155d None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.525198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77349f8a-e19a-4483-8174-14f0ab0f155d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.528211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 198/400] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:09 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:09.530010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.530463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.530879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.539091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:09.548365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45edbc7f-d175-46c1-b28f-e88f6f7fe3dc None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.553268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:09.555217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:09.589501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaaf296-d24b-4cdb-a58e-02c6a1b8b2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.590304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaf296-d24b-4cdb-a58e-02c6a1b8b2f4 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.590585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaf296-d24b-4cdb-a58e-02c6a1b8b2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.590855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaaf296-d24b-4cdb-a58e-02c6a1b8b2f4 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/groups/d78c75e6dc9549e9b0b744039a254e55/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.592868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaf296-d24b-4cdb-a58e-02c6a1b8b2f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3d1f0afb5114fc887e7a1565d3707b0, group_id=d78c75e6dc9549e9b0b744039a254e55, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.594333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaaf296-d24b-4cdb-a58e-02c6a1b8b2f4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.610505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 203/401] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 05:33:09 2022] PUT /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/groups/d78c75e6dc9549e9b0b744039a254e55/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:09.612094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.612618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.613361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.638887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:09.642358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:09.647769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 199/402] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:09.655091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.692282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:09.809416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 204/403] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:09.865981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caff92b6-d7e9-45bc-b7df-e84c3e5685af None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:09.905989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:09.907165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:09.907669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:09.908170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:09.910402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:09.911937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:09.916877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 200/404] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:09 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:09.920424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.984642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:09.987609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:10.055463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3c15a-1ee4-4775-b74e-54f62677bc3f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.090769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f741e2d-93d4-46eb-89a4-eb401c6b46b3 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.113450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.114155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.114433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.114702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/groups/d78c75e6dc9549e9b0b744039a254e55/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.116850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] RBAC: Authorizing `identity:create_grant(project_id=68f16fe3336647ac92f5dc9a178bc2d4, group_id=d78c75e6dc9549e9b0b744039a254e55, role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.118328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.128955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.129818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.130204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.131350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.144908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:10.145449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 205/405] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 05:33:09 2022] PUT /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4/groups/d78c75e6dc9549e9b0b744039a254e55/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:33:10.146495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:10.151115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:10.152243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:10.153140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:10.154321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 201/406] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:10.160439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.163554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:10.232266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:10.266208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29d0ce7-ec20-4e9b-9ed2-d4f32e762ceb None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.294787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e96c26-2b37-436f-b473-0f62dfdefee9 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.308517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.308517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.308517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.308881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.321413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:10.328258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:10.329130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:10.330366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 206/407] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:10.336501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.348283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.348878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.349163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.349544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.350324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:10.350678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.352174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.363564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 202/408] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:10.368900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.441762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:10.503816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28316c38-8d01-48fa-8c23-1f170094ab6a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.546330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.547109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.547489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.547861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.548699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:10.549122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.550664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.555369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24844b31-6f40-44e0-9248-524044d61838 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.560718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 207/409] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:10.566162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.602077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:10.625823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.626529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.626806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.627073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.634398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.635965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.637871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 203/410] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:10.643734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.677136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf8c6e5-d884-4499-98fa-5cbb435a0f36 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.713043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.713809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.714201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.714546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] PATH_INFO: `/v3/groups/9edf96c20b3841dab30f060739a836ca` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.715753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] RBAC: Authorizing `identity:get_group(group_id=9edf96c20b3841dab30f060739a836ca)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.717262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.719342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 208/411] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/groups/9edf96c20b3841dab30f060739a836ca => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:10.724671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.727439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:10.862010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d8c958-bb55-46a9-9623-6078ca966a71 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.872025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec64a612-39f8-41a8-b23e-52bfefdae515 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:10.910427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.912230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.915389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.915890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] PATH_INFO: `/v3/projects/136c4807ccbf43c49f1d29adcff44403` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.917363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] RBAC: Authorizing `identity:get_project(project_id=136c4807ccbf43c49f1d29adcff44403)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.919116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.921104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 209/412] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/projects/136c4807ccbf43c49f1d29adcff44403 => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:10.930908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:10.959282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:10.959924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:10.960200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:10.960468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:10.966301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:10.967761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:10.973587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 204/413] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:10 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:10.975823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:11.026227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:11.070235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa852a5-75c5-45d1-85ef-4a6c6391a638 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:11.079444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:11.140523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410523ef-696a-4383-bac5-014056ac4306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:11.141686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410523ef-696a-4383-bac5-014056ac4306 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:11.143186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410523ef-696a-4383-bac5-014056ac4306 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:11.143773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410523ef-696a-4383-bac5-014056ac4306 None admin] PATH_INFO: `/v3/projects/136c4807ccbf43c49f1d29adcff44403/groups/9edf96c20b3841dab30f060739a836ca/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:11.147821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410523ef-696a-4383-bac5-014056ac4306 None admin] RBAC: Authorizing `identity:create_grant(project_id=136c4807ccbf43c49f1d29adcff44403, group_id=9edf96c20b3841dab30f060739a836ca, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:11.149647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410523ef-696a-4383-bac5-014056ac4306 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:11.161783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61f1027-17f9-4f73-b9b5-2deba4e1ea68 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:11.173782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 210/414] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 05:33:10 2022] PUT /identity/v3/projects/136c4807ccbf43c49f1d29adcff44403/groups/9edf96c20b3841dab30f060739a836ca/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:11.221098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ac372-6d5e-40f1-be69-10a043259850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:11.221793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ac372-6d5e-40f1-be69-10a043259850 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:11.222072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ac372-6d5e-40f1-be69-10a043259850 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:11.222341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ac372-6d5e-40f1-be69-10a043259850 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/groups/d78c75e6dc9549e9b0b744039a254e55/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:11.224492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ac372-6d5e-40f1-be69-10a043259850 None admin] RBAC: Authorizing `identity:create_grant(project_id=b3d1f0afb5114fc887e7a1565d3707b0, group_id=d78c75e6dc9549e9b0b744039a254e55, role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:11.225957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ac372-6d5e-40f1-be69-10a043259850 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:11.244173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 205/415] 10.222.0.73 () {58 vars in 1442 bytes} [Wed Aug 24 05:33:10 2022] PUT /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0/groups/d78c75e6dc9549e9b0b744039a254e55/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:33:11.577947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20f3e5-c8ae-4d8d-8b61-e04d0dd82ad1 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:11.578330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20f3e5-c8ae-4d8d-8b61-e04d0dd82ad1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:11.578679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20f3e5-c8ae-4d8d-8b61-e04d0dd82ad1 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:11.580322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 211/416] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:11.585948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e22443-4ae4-4fe5-86f0-ffa75ad94b4f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:11.586444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e22443-4ae4-4fe5-86f0-ffa75ad94b4f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:11.586655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e22443-4ae4-4fe5-86f0-ffa75ad94b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:11.605779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-80e22443-4ae4-4fe5-86f0-ffa75ad94b4f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:11.606983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-80e22443-4ae4-4fe5-86f0-ffa75ad94b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:11.704166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 206/417] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:11.984763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c88646-ebf4-4986-8f7f-16c1a2d17c91 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:11.985108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c88646-ebf4-4986-8f7f-16c1a2d17c91 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:11.985281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c88646-ebf4-4986-8f7f-16c1a2d17c91 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:11.987039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 212/418] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:11.992424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:11.992712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:11.992982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.016558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:12.017785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:12.038620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c88646-ebf4-4986-8f7f-16c1a2d17c91 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:12.102006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:12.144354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 207/419] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:12.191370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c88646-ebf4-4986-8f7f-16c1a2d17c91 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:12.229349 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:12.239575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:12.240152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.240495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.240738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.263823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:12.269138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:12.270108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:12.271368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 213/420] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:12.276070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:12.377142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79e887c-2540-4cb2-b35f-7008bf1336c1 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:12.411820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:12.412653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.413074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.413538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.414434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:12.414878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:12.416485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:12.425519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 208/421] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:12.430642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:12.500053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:12.567192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea10781-0661-4ebe-a401-ebb69c9e94ea None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:12.600035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:12.606782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:12.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.607682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.607998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.695876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:12.700104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:12.701600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:12.753487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5e9c0d-6c84-477f-89bd-7b7ce1c19f23 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:12.809446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 214/422] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:33:12.815494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:12.827112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b922911-65d1-4b70-9b2a-d8f8fa3037b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:12.827772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b922911-65d1-4b70-9b2a-d8f8fa3037b5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.828050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b922911-65d1-4b70-9b2a-d8f8fa3037b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.828317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b922911-65d1-4b70-9b2a-d8f8fa3037b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.850652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b922911-65d1-4b70-9b2a-d8f8fa3037b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2b922911-65d1-4b70-9b2a-d8f8fa3037b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:12.868291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:12.869736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:12.870536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 209/423] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:12.877354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078934e-7005-45c5-a062-af940ba02fab None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.877658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078934e-7005-45c5-a062-af940ba02fab None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.877932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e078934e-7005-45c5-a062-af940ba02fab None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.879319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 210/424] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:33:12.879876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078934e-7005-45c5-a062-af940ba02fab None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:12.925790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:12.937556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2fa32f-f6b1-4e4a-a8e0-1abf1a01c733 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:12.976720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dab259-2f7a-4a02-8f10-6dff8949f5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:12.977412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dab259-2f7a-4a02-8f10-6dff8949f5e7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.977739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dab259-2f7a-4a02-8f10-6dff8949f5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.978015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dab259-2f7a-4a02-8f10-6dff8949f5e7 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:12.979423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dab259-2f7a-4a02-8f10-6dff8949f5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:12.981207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dab259-2f7a-4a02-8f10-6dff8949f5e7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:12.987510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 215/425] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:12 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:12.987510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:12.987510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:12.987510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.005048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:13.006276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:13.023510 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e078934e-7005-45c5-a062-af940ba02fab None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.028020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:13.068353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.069074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.069450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.069828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.070578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:13.070953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:13.074023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:13.085808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 211/426] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:33:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:13.086792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:13.087709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 216/427] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:13.098863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:13.224618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9619cffc-05cb-45d1-a991-7b65b19bdda3 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.261462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.262613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.263154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.263636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.265053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:13.266797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:13.268610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 212/428] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:13.274471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:13.277944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:13.298428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d81c610-3d83-4365-a377-efdc29daf45e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.341407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.342094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.342731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.343012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.344073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:13.426982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:13.428493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:13.509166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff637316-a086-428c-8104-8f96d3bc1e0c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.548394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.548927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.549206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.549480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.550612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:13.550930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:13.552463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:13.560791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 213/429] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 611 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:13.561703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:13.569396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:13.586790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 217/430] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 05:33:13.595024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:13.723027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:13.732417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb80120-2e89-43f4-a07e-412bddc1369e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.744340 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bc2df3-f537-4636-8686-768e3b267590 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.777707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.778372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.778653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.778931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.784928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5e5999-4cc5-45a8-9fab-2664b9255124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.785857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5999-4cc5-45a8-9fab-2664b9255124 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.786341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5999-4cc5-45a8-9fab-2664b9255124 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.786608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5e5999-4cc5-45a8-9fab-2664b9255124 None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.787726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e5999-4cc5-45a8-9fab-2664b9255124 None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:13.789148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5e5999-4cc5-45a8-9fab-2664b9255124 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:13.790834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 218/431] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:13.791710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:13.792811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347039f1-af77-4a84-ba65-84983e4a7676 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.793214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347039f1-af77-4a84-ba65-84983e4a7676 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.793487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347039f1-af77-4a84-ba65-84983e4a7676 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.794766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 219/432] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:13.795790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f6af6-4b66-420d-bb16-4e3a2e42565d None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.796125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f6af6-4b66-420d-bb16-4e3a2e42565d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.796637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f6af6-4b66-420d-bb16-4e3a2e42565d None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:13.799372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:13.800245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:13.801260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 214/433] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:13.802315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:13.802379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 220/434] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:13.803613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b794b74-ddce-439e-939c-5a497d5440d3 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.803613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b794b74-ddce-439e-939c-5a497d5440d3 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.803613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b794b74-ddce-439e-939c-5a497d5440d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.822017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4b794b74-ddce-439e-939c-5a497d5440d3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:13.823750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4b794b74-ddce-439e-939c-5a497d5440d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:13.885937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:13.906885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 221/435] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:13.908432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.908887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.909784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.923605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a728bc90-dc2c-4e9a-83ce-e8ccb078d6be None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:13.931450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:13.932690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:13.962681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:13.964367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:13.964367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:13.965162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:13.968023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:13.969462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:13.971221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 215/436] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:13.971816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.007519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.015366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 222/437] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:14.016044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.110600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b041bfb-391b-4ce4-94ab-32cdeac7e255 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.143173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23b928-2e8c-4341-84a0-3a230e1ea145 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.147758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.148371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.148651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.148926 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.149683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:14.149995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.152461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.162135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.168691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 216/438] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:14.169607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.181136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.182372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.183242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.183587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.184385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=e6fe411570314ffdb43c23bed35b1f50, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:14.184761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.187088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.193685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 223/439] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=e6fe411570314ffdb43c23bed35b1f50 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:14.194244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.288006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.308994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acdf62-ac72-4148-b2d3-7c7e7c81271f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.332835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef8664-e10c-4962-aafe-9e426175de15 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.347821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.348440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.348721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.348996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.376295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.376931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.377131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.377401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.392517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:14.398989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:14.400052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:14.401220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 224/440] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:14.401930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.428282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.429776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.447375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.505965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfdcff-44e4-476e-a07a-7df21d1f7321 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.514831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.530182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 217/441] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:14.531005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.540129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.540762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.541576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.541854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.552580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:14.558475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:14.559301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:14.560083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 225/442] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:14.560714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.635360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf21ae30-4b03-428b-ad49-0d781c758777 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.658125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.661592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07587a-a6d5-41da-b63a-1670a7edc947 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.669343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.669971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.670252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.670527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.671191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:14.671484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.672847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.680736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 218/443] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:14.681512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.696470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.697101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.697924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.698200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] PATH_INFO: `/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.699724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] RBAC: Authorizing `identity:get_project(project_id=68f16fe3336647ac92f5dc9a178bc2d4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.701161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.702790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 226/444] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/projects/68f16fe3336647ac92f5dc9a178bc2d4 => generated 298 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:14.703581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.731255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.781237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2af1169-6bc4-46de-b8f6-a16496645186 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.801398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f91b3-b73e-499b-919a-c83ea64797f1 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.814505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.815107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.815386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.815664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.816314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:14.816613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.818031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.826298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 219/445] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:14.830467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:14.834152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.834755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.835028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.835295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.869078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.904139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.905518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.932215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05137798-4144-493a-81e8-22c9a999f75e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:14.945390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:14.969696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:14.970333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:14.970612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:14.970888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:14.972008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:14.973455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:14.975051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 220/446] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:14.975821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.009367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 227/447] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:15.010787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.077028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d664a60-1039-4775-ac05-6c5c576c56cf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.079169 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:15.109976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.110631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.110918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.111195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.112057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b854ea3-4347-4ecb-af08-eb7b43d01b1b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.146170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.146779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.147062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.147336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.148055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:15.148363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.149810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.155941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 228/448] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=68f16fe3336647ac92f5dc9a178bc2d4&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:15.156670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.171530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:15.200648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.202464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.275523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf130e-24be-401d-85e0-939f49550c95 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.311193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:15.319297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.319297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.319297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.319593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] PATH_INFO: `/v3/projects/136c4807ccbf43c49f1d29adcff44403` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.321436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] RBAC: Authorizing `identity:get_project(project_id=136c4807ccbf43c49f1d29adcff44403)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.322249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.323163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 221/449] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:15.323893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 229/450] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/projects/136c4807ccbf43c49f1d29adcff44403 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:15.328882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.330207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.421957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:15.454268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f4d508-deb3-4f91-872e-e82c7c7e36ee None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.456165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709214a7-f8fe-41ba-81ee-de75c0f8d79c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.494809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.495281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.495567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.495839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] PATH_INFO: `/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.497021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] RBAC: Authorizing `identity:get_project(project_id=b3d1f0afb5114fc887e7a1565d3707b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.498816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.498935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.499007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.499007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.499143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] PATH_INFO: `/v3/groups/9edf96c20b3841dab30f060739a836ca` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.501605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 222/451] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/projects/b3d1f0afb5114fc887e7a1565d3707b0 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:15.501723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] RBAC: Authorizing `identity:get_group(group_id=9edf96c20b3841dab30f060739a836ca)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.501723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.503662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 230/452] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/groups/9edf96c20b3841dab30f060739a836ca => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:15.505975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.510131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.583007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:15.622023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66182f3-9c1f-4c4b-a111-9bf648501b12 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.629721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e434d77-155e-4024-b8dd-da7fd97fa75c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.657811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.658428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.658707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.658978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.659703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] RBAC: Adding query filter params (group.id=9edf96c20b3841dab30f060739a836ca, role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=136c4807ccbf43c49f1d29adcff44403) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:15.660008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.662573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.669249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 231/453] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/role_assignments?group.id=9edf96c20b3841dab30f060739a836ca&scope.project.id=136c4807ccbf43c49f1d29adcff44403&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:15.670196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1d69-b485-4d3c-861c-168f2a89e441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.670825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1d69-b485-4d3c-861c-168f2a89e441 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.671101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1d69-b485-4d3c-861c-168f2a89e441 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.671369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1d69-b485-4d3c-861c-168f2a89e441 None admin] PATH_INFO: `/v3/groups/d78c75e6dc9549e9b0b744039a254e55` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.672557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1d69-b485-4d3c-861c-168f2a89e441 None admin] RBAC: Authorizing `identity:get_group(group_id=d78c75e6dc9549e9b0b744039a254e55)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.674051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff1d69-b485-4d3c-861c-168f2a89e441 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.675711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 223/454] 10.222.0.73 () {56 vars in 1180 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/groups/d78c75e6dc9549e9b0b744039a254e55 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:15.682482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:15.786797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69d9cc-551f-4a49-bfb5-1288cc9bd636 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:15.817495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:15.823149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:15.823766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:15.824096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:15.824404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:15.825023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] RBAC: Adding query filter params (group.id=d78c75e6dc9549e9b0b744039a254e55, role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:15.825324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:15.826816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dfaf7e-3631-49bb-b592-187f0d8d763b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:15.832993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 232/455] 10.222.0.73 () {56 vars in 1376 bytes} [Wed Aug 24 05:33:15 2022] GET /identity/v3/role_assignments?group.id=d78c75e6dc9549e9b0b744039a254e55&scope.project.id=b3d1f0afb5114fc887e7a1565d3707b0&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:18.347560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8a41-a895-4d5b-85fc-0e17a0558005 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:18.348428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8a41-a895-4d5b-85fc-0e17a0558005 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:18.348841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6d8a41-a895-4d5b-85fc-0e17a0558005 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:18.350569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 224/456] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:18.356741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19231785-bcd3-4db0-8681-65c257d03c60 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:18.357168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19231785-bcd3-4db0-8681-65c257d03c60 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:18.357379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19231785-bcd3-4db0-8681-65c257d03c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:18.382154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-19231785-bcd3-4db0-8681-65c257d03c60 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:18.383359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-19231785-bcd3-4db0-8681-65c257d03c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:18.472326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 233/457] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:18.486127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6bb0c-79da-4874-88ab-1c0652ac1c0f None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:18.486713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6bb0c-79da-4874-88ab-1c0652ac1c0f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:18.487180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d6bb0c-79da-4874-88ab-1c0652ac1c0f None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:18.488922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 225/458] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:18.493777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296524a1-f5ae-4340-8d9e-d15681b50bd1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:18.494111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296524a1-f5ae-4340-8d9e-d15681b50bd1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:18.494359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296524a1-f5ae-4340-8d9e-d15681b50bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:18.520510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-296524a1-f5ae-4340-8d9e-d15681b50bd1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:18.521774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-296524a1-f5ae-4340-8d9e-d15681b50bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:18.624178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 234/459] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:18.646765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b0fb7-fc23-4209-ab5f-dcd8f446ccf2 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:18.647350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b0fb7-fc23-4209-ab5f-dcd8f446ccf2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:18.647834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b0fb7-fc23-4209-ab5f-dcd8f446ccf2 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:18.649610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 226/460] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:18.655486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:18.655770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:18.656044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:18.682178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:18.683392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:18.755967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:18.778817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 235/461] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:18.888519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b0fb7-fc23-4209-ab5f-dcd8f446ccf2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:18.921410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.059469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b0fb7-fc23-4209-ab5f-dcd8f446ccf2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.104931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.105464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.105663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.105932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.106206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.118523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.120598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:19.140779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:19.142030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:19.142854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:19.142854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:19.142854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:19.142854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 227/462] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:19.143099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.224964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.231472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e4e046-c75e-4515-93d7-dca54ee0c5bb None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.266202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567b0de-41d5-426e-8664-2cd4e178a4eb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.273502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.274623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.274623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.274993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.287620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.289315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.302580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.303216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.303500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.303774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:19.312190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:19.313227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:19.314221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:19.314221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:19.314221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:19.319941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 236/463] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:19.320082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.320161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.320228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-a1d7d545-5b8f-40b8-b47b-1293c6994a4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:19.335798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:19.336737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:19.337555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:19.337555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:19.337555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:19.337555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 228/464] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:19.337555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373e55c-1ebd-4370-9c49-e24dc67c26aa None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.338941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373e55c-1ebd-4370-9c49-e24dc67c26aa None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.339163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373e55c-1ebd-4370-9c49-e24dc67c26aa None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.344923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 229/465] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:19.344923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373e55c-1ebd-4370-9c49-e24dc67c26aa None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.367567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.433044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4ed6a3-950e-4d81-83ab-ab4e420e660c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.455562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373e55c-1ebd-4370-9c49-e24dc67c26aa None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.475012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.475570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.476015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.476482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.476743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.477390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:19.477737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.479389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5397dfe-17fd-44f4-8294-df4453f250dc None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.491718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 237/466] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:19.492913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.493194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.493470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.502512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.503409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.503930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.504391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.505220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:19.505724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.508015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.511752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:19.512996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:19.524347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 230/467] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:19.526363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.580959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 238/468] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:19.581719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.611120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.664636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b0251-732f-4ac9-ba80-71af669967ea None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.707263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.707984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.708249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.708641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.709370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:19.709774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.711073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0d9b9-e65c-4ca6-a2fb-eb211c5c367d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.711233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.729672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 231/469] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:19.729672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.757726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.758389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.758670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.758942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.759581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.761149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.779860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.793588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 239/470] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:33:19 2022] POST /identity/v3/users => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:19.793588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.867293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832fb7a5-3152-4cef-b8ae-e936e103d8a5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.894787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:19.907977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c015154-3856-48ba-8a6f-a38d17d626c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.908590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c015154-3856-48ba-8a6f-a38d17d626c9 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.908872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c015154-3856-48ba-8a6f-a38d17d626c9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.909146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c015154-3856-48ba-8a6f-a38d17d626c9 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.909833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c015154-3856-48ba-8a6f-a38d17d626c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.911304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c015154-3856-48ba-8a6f-a38d17d626c9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:19.916555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee1bce-063b-442b-874b-13b26cc48580 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:19.936999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 232/471] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:33:19 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:19.938436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8774da-bcd2-4a81-b990-1b3860b62d79 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.938793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8774da-bcd2-4a81-b990-1b3860b62d79 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.939130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8774da-bcd2-4a81-b990-1b3860b62d79 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.945673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 233/472] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:19.945673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8774da-bcd2-4a81-b990-1b3860b62d79 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:19.974737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b66eb1-6a6c-4cae-b0ee-8dc80e0c8244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:19.975819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b66eb1-6a6c-4cae-b0ee-8dc80e0c8244 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:19.976143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b66eb1-6a6c-4cae-b0ee-8dc80e0c8244 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:19.976374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b66eb1-6a6c-4cae-b0ee-8dc80e0c8244 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:19.977415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b66eb1-6a6c-4cae-b0ee-8dc80e0c8244 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:19.979135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b66eb1-6a6c-4cae-b0ee-8dc80e0c8244 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:20.010175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 240/473] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:33:19 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:20.012849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:20.012849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:20.012849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:20.035943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:20.037547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:20.109959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:20.110918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8774da-bcd2-4a81-b990-1b3860b62d79 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:20.142333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 241/474] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:20.161445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56106548-59cd-428a-8903-20a56820574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:20.162156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56106548-59cd-428a-8903-20a56820574e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:20.162442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56106548-59cd-428a-8903-20a56820574e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:20.162720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56106548-59cd-428a-8903-20a56820574e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:20.175558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56106548-59cd-428a-8903-20a56820574e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:20.177121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56106548-59cd-428a-8903-20a56820574e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-56106548-59cd-428a-8903-20a56820574e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:20.196907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:20.197906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:20.198717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:20.198717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:20.198717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:20.198717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 234/475] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:20.203873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:20.281809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:20.317692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fd17c9-c6e5-4f78-b9df-eb700e087d28 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:20.357972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:20.358592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:20.358873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:20.359146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:20.359802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:20.360101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:20.361471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:20.374449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 242/476] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:20.381841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56106548-59cd-428a-8903-20a56820574e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:20.446958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:20.523910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56106548-59cd-428a-8903-20a56820574e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:20.585741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:20.586399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:20.586678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:20.586951 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:20.587781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:20.589785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:20.592683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c007e0-2592-479a-9e0e-85bf4ee7c1ad None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:20.626153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:20.633402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 235/477] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:33:20 2022] POST /identity/v3/users => generated 249 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:20.633939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:20.635459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:20.635970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:20.636440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:20.637328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:20.639215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:20.654760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.sql.core [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:33:20.654760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 05:33:20.654760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [parameters: {'id': '7673e0b33cf143e2920346c60a198796', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 05:33:20.654760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result.read() Aug 24 05:33:20.679223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:33:20.680139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application yield resource Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise self.value Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application session.commit() Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:33:20.681044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:33:20.682582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:33:20.683446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result.read() Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:33:20.685875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [parameters: {'id': '7673e0b33cf143e2920346c60a198796', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:20.687884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 05:33:20.689912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:20.697846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 243/478] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 05:33:20 2022] POST /identity/v3/roles => generated 144 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:20.700077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:20.703496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:20.815044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f3d180-b8bf-4f6c-8040-3771278ae2c7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:20.851653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ce43e3-aca3-497d-8c97-d804bef13db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:20.852654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce43e3-aca3-497d-8c97-d804bef13db5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:20.853129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce43e3-aca3-497d-8c97-d804bef13db5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:20.853908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce43e3-aca3-497d-8c97-d804bef13db5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:20.864931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ce43e3-aca3-497d-8c97-d804bef13db5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-60ce43e3-aca3-497d-8c97-d804bef13db5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:20.870733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:20.871605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 05:33:20.872371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:20.877590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 236/479] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:33:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:20.879397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:20.991010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7b9146-3b0f-4cff-812c-443dc441f513 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:21.029515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:21.030529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:21.031010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:21.031477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:21.032339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:21.032849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:21.034471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efab571-9336-4177-a576-0a3c82db72ad None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:21.043521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 244/480] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:22.356179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d55b-5d65-40cf-9fe7-5cdf05f8f2a6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:22.357156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d55b-5d65-40cf-9fe7-5cdf05f8f2a6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:22.357649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14d55b-5d65-40cf-9fe7-5cdf05f8f2a6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:22.359466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 237/481] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:22.364535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f30b64-82f8-463f-ab0b-f624229c1efa None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:22.364924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f30b64-82f8-463f-ab0b-f624229c1efa None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:22.365089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f30b64-82f8-463f-ab0b-f624229c1efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:22.390640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-11f30b64-82f8-463f-ab0b-f624229c1efa None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:22.391831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-11f30b64-82f8-463f-ab0b-f624229c1efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:22.402718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8eb4d-228c-4a81-8f4c-7ccaaad31a23 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:22.403281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8eb4d-228c-4a81-8f4c-7ccaaad31a23 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:22.403773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf8eb4d-228c-4a81-8f4c-7ccaaad31a23 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:22.405547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 238/482] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:22.410746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24374e8e-a6ad-4c61-a0eb-fbab8ccbd0ed None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:22.411026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24374e8e-a6ad-4c61-a0eb-fbab8ccbd0ed None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:22.411294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24374e8e-a6ad-4c61-a0eb-fbab8ccbd0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:22.441270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-24374e8e-a6ad-4c61-a0eb-fbab8ccbd0ed None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:22.442484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-24374e8e-a6ad-4c61-a0eb-fbab8ccbd0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:22.493145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:22.517240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 245/483] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:22.558077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:22.576052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 239/484] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:22.685388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27ffa-0e09-439c-bea1-2e1fb97c48a1 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:22.685750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27ffa-0e09-439c-bea1-2e1fb97c48a1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:22.685970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d27ffa-0e09-439c-bea1-2e1fb97c48a1 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:22.687622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 246/485] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:22.692845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:22.693141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:22.693418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:22.712789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:22.714151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:22.802774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 240/486] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:22.850460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d27ffa-0e09-439c-bea1-2e1fb97c48a1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:22.903385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.006305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d27ffa-0e09-439c-bea1-2e1fb97c48a1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.063095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104998f-e9f3-409b-82e0-14b82ed40af2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.080412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.081047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.081320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.081593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.102032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.107366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:23.112129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:23.112981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:23.113816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 247/487] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:23.114444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.115027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.115122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.115331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.127715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.136038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:23.141203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:23.142104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:23.142865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:23.143926 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 241/488] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:23.149860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.212499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.260063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03d5eb-0c93-4a8e-a7ea-eed2e07c4ac3 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.277009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c760f4-1a99-4ce3-8534-3225b4ee015a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.301072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.301708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.301993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.302266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.302931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:23.303238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.304612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.312875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 242/489] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:23.313711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.319157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.319776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.320055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.320327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.320996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:23.321297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.322793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6d2ee-44a0-42ae-8983-28c54578d825 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.331242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 248/490] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:23.331242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55bc72-6b91-4e33-8e69-785768d69e29 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.331242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55bc72-6b91-4e33-8e69-785768d69e29 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.331242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55bc72-6b91-4e33-8e69-785768d69e29 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.331242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 249/491] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:23.331242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55bc72-6b91-4e33-8e69-785768d69e29 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.370925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.428235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b3b7f-152c-4e1c-aaa6-a20933362479 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.447646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55bc72-6b91-4e33-8e69-785768d69e29 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.458222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.461768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.462411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.463179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.463459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.473629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:23.479408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:23.480330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:23.481106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 243/492] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:23.481802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.487297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.487927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.488209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.488477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.499647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.501077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-905034f0-f733-480d-a867-0728a8ac3223 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:23.516730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 250/493] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8460fb-9df6-4b8d-97e9-228ed64f0df2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8460fb-9df6-4b8d-97e9-228ed64f0df2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.520039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8460fb-9df6-4b8d-97e9-228ed64f0df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.536830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3e8460fb-9df6-4b8d-97e9-228ed64f0df2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:23.538117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3e8460fb-9df6-4b8d-97e9-228ed64f0df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:23.584005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326dfe3-575a-4f21-a8c4-537195a8056c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.586289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.606083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 251/494] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:23.607314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b97f73-4e4e-4d90-a604-e9e335cfb49a None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.607592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b97f73-4e4e-4d90-a604-e9e335cfb49a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.607853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b97f73-4e4e-4d90-a604-e9e335cfb49a None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.609305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 252/495] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:23.610013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b97f73-4e4e-4d90-a604-e9e335cfb49a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.626530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.627076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.627355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.627886 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.638864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.640327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:23.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:23.656081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:23.656925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:23.656925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:23.656925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:23.657082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 244/496] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:23.657969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.723196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.736386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b97f73-4e4e-4d90-a604-e9e335cfb49a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.768909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5cc3a-fb62-413f-aa34-9f03ffbf1911 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.772486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.773376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.773696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.773973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.774636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:23.774940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.776959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdfd44-658b-46ab-824b-eea2dcd3e8ff None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.785321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 253/497] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:23.790921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.791241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.791526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.805513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.806144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.806455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.806992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.807372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:23.807674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.808687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:23.809059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.809904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:23.823565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 245/498] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:23.824387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.846306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.888996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 254/499] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:23.889780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:23.931968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556546e5-c38b-441f-9dfb-5ba63f245ec9 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:23.969925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:23.971067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:23.971771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:23.972193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:23.973132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:23.973858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:23.974065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:23.975949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:23.992118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 246/500] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:23.992944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.002177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5408d52-7dbf-41fc-a8a9-1177bc21347c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.040953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.041681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.041962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.042226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.052683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.054762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:24.071465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:24.073220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:24.074406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:24.074406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:24.074406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:24.078441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 255/501] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:24.079308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.100039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.127199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc04f5a-67ac-476d-b551-61d41b8a6cec None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.163912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.164455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.164729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.164993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.172502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.173957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.177635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8a4bd-eb8f-4ed6-b941-22fffaec7313 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:24.186481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:24.187567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 247/502] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:24.188836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.215173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.215939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.216293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.216629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.218623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.228141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:24.233630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:24.234655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 256/503] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:24.235456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.292735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1df99a-e5fe-498b-bb71-005e1269ebc1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.325545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.326177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.326453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.326721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.329643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.334911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.336957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.337542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4915f16-782a-473f-ba3c-7778a61a5d1d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:24.348661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:24.349537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:24.350444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 248/504] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:24.350987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.379437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.380045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.380325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.380604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.381242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:24.381549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.382952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.394894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 257/505] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:24.395726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.452725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.462924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c367a809-9937-459a-9c64-825843bb01e2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.497501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.497726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276906e0-1624-4668-9b6f-8a0cb04c4585 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.498153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.498434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.498711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.510571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.512035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:24.526028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:24.526916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:24.527735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:24.527735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:24.527735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:24.527735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 249/506] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:24.527946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.534498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.535108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.535391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.535667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.536397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:24.536704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.538218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.577038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 258/507] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:24.577921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.583340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.635019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b094ab34-b099-4b8e-8b3d-495a025a2466 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.669521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.676394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.676394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.676394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.676394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:24.676394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.676394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.681254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91f0ae-5a98-4741-b535-003243d4c058 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.683843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 250/508] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:24.684648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.699339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.721402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.721955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.722227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.722491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.723541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:24.723840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.725272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.742360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 259/509] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:24.743183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.795736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadff7b-394d-4a2c-acaa-18e419d09d2f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.805897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.830462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.831114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.831385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.831650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.839034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.840668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.842798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddc6de-b5e4-42fd-9efe-5b215f09654e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:24.851572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:24.852432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:24.853338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 251/510] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:24.854176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.879794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.880397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.880675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.880952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:24.881610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:24.881914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:24.883285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:24.895199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 260/511] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:24.896143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:24.926465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:24.962216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76d525-9958-4d8d-ab87-5b9c0585d2cc None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.995443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:24.995974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:24.996070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc24c4-2817-49b4-80e1-28bdf04391cc None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:24.996253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:24.996523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.029852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.030485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.030783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.031112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.036176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.049090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.050694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:25.067202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:25.068160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:25.068961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:25.068961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:25.068961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:25.068961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 261/512] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:25.069175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.069244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.069867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.152603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.170115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163e784-ae5b-4393-803c-c93bb867ede4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.170876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 252/513] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:25.171711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.207412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.208009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.208286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.208558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.240014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.271395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d839b0d9-5cb3-4854-983b-ddb9b4037c0e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.282157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.283616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.305630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.306233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.306514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.306785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.307658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:25.307809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.309224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.351113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 253/514] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:25.351943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.363662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.385728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 262/515] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:25.386510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.458282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6aad02-4c33-4a9d-abc0-ce8cb98f3667 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.483910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.484680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4f470b-d783-495c-9309-3c0b94ab6f1e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.493260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.494512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.494792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.495072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.495723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.497207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.518368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 254/516] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:33:25 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:25.519185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.523006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.523541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.523872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.524152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.524807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:25.525146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.527518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.545294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 263/517] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:25.545294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.595217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.639416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed39e6-86a1-4d3d-b158-f55f3cd48976 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.670979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11482887-17d2-4f68-a428-8b6255278f6a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.680957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.682161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.682442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.682714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.699628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.700965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.714022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.714634 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.714913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.715185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:25.717399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:25.718486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:25.719533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:25.719533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:25.719533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:25.719726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 255/518] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:25.720943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.758145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.795333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.796702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.826961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c87b3-c275-49b7-94db-7fea96aaf3f2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:25.839431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:25.865713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:25.866352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:25.866659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:25.866894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:25.879770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:25.881996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:25.900722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:25.901771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:25.902604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:25.902604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:25.902604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:25.902604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 256/519] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:25.902808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:25.909390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 264/520] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:25.910499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.002434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.016762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a4a9e-3a60-4df0-977a-cdb7881153ea None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.017201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcdb9dc-d18b-478c-9100-6927d911b87a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.052749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.053280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.053375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.053605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.053860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.054310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.054501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:26.054844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.054941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.055270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.056374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.064349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.065991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.068058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 265/521] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:26.069162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.071869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:26.082558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:26.083660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:26.084567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 257/522] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:26.085106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.180950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffca4c7-5d02-40be-bb3c-838bba75a5e5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.201827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604d67-7084-4e40-9e9b-943e9736c142 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.219644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.220252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.220523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.220787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.221418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:26.221751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.223104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.235352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 266/523] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:26.236019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.245795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.246832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.247114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.247387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.262602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.264313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.274346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:26.281759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:26.282811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:26.283989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:26.283989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:26.283989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:26.290680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 258/524] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:26.291464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.304196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.351767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdedac-88f9-466a-9f76-60ef38dd9f5e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.390435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.391057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.391335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.391610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.395828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1866241b-219a-4407-9cfa-15f5ff9b63e1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.398345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.399788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:26.412464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:26.413306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 267/525] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:26.414591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.437380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.438020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.438297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.438563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.439278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:26.439795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.441003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.459550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 259/526] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:26.459550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.552092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.552708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c63a8c7-adc0-45c6-89b8-9fd286203b02 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.588930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.606838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.607809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.608239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.608697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.619236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.620071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df332505-9fa3-46b9-b578-aa5e2931000e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.621809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:26.642395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:26.643333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:26.644184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 268/527] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:26.645241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.676593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.677453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.677921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.678196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.678935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:26.679243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.680614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.694406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 260/528] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:26.695600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.758871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde8f6f-79c4-4196-bc4f-b7cf7c27d82d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.793739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.794633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.795094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.795511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.796337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:26.796784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.797984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2d4b6f-27df-44af-b8b2-8a27cf8b0c02 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.798583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.817192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 269/529] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:26.818049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:26.822848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.843169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.844008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.844291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.844563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.865395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:26.917430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.918962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.927947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e205e56-5a54-4037-87ce-04284b41f4c2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:26.964794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:26.965783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:26.966257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:26.966855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:26.967876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:26.968441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:26.970192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:26.984923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 270/530] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:26.985761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.030522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 261/531] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:27.035871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.054314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:27.095776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e520f69-6ac2-4bf4-9552-13a8984e407a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.098888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:27.141315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.142033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.142319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.142589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.149470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.150608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2f2283-e86c-475a-a555-cfd3abd3ebaf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.151012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:27.163387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:27.164395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:27.165329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 271/532] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:27.166198 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.193753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.194494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.194869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.195231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.196047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=960468d880f34ea2a267b6ada5fb30bd) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:27.196451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.198441 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.204209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 262/533] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/role_assignments?user.id=960468d880f34ea2a267b6ada5fb30bd&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:27.204988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.309474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e659c-ab21-472f-938e-a89bf9a38eb5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.324628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:27.351200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206412-eb11-4685-8865-b21170703234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.351594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35327166-e976-421b-b9d3-fba5aee51795 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.352246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206412-eb11-4685-8865-b21170703234 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.352660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206412-eb11-4685-8865-b21170703234 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.354170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87206412-eb11-4685-8865-b21170703234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.356101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206412-eb11-4685-8865-b21170703234 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=4e926d4a438e42ecaeab44f711e0b3c9) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:27.356593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206412-eb11-4685-8865-b21170703234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.358231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87206412-eb11-4685-8865-b21170703234 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.366160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 272/534] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/role_assignments?user.id=4e926d4a438e42ecaeab44f711e0b3c9&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:27.367538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206412-eb11-4685-8865-b21170703234 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.411190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:27.417675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.418987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.419266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.419553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.451549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.453049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-8a347910-0a0d-4987-bf51-1f0b0c0de666 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:27.484094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:27.485056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:27.486331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:27.486331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:27.486331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:27.486331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 263/535] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:27.486773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5bdd0-477c-42cf-9cca-3ceae1ce77c9 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.487053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5bdd0-477c-42cf-9cca-3ceae1ce77c9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.487321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5bdd0-477c-42cf-9cca-3ceae1ce77c9 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.493300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 264/536] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:27.494386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5bdd0-477c-42cf-9cca-3ceae1ce77c9 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.519279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87206412-eb11-4685-8865-b21170703234 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.565923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.566654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.566936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.567207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.568012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:27.568321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.569805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb19a62-ee33-4a7b-ba2f-acbf96b34ddb None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.589384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 273/537] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:27.589384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.589384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.589384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.611061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:27.612620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:27.625383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5bdd0-477c-42cf-9cca-3ceae1ce77c9 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.669677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.671055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.671800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.672244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.673993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:27.674439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.675965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.678275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:27.700037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 265/539] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:27.700037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.700456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 274/539] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:27.701509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.747554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:27.850984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb20f95-2f6a-474c-94ff-85e77ee2c57b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.913205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.913758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.914152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.914426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.915161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=1c7f12a5dc6446eaa94a96dbb44ccdaf) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:27.915468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.917543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.923797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 266/540] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/role_assignments?user.id=1c7f12a5dc6446eaa94a96dbb44ccdaf&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:27.925881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a28e7-e812-4d71-aad0-734bc482f207 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:27.966909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:27.967766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:27.968163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:27.968537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:27.977089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:27.979138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:27.987928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:27.994615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:27.995559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:27.996522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 275/541] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:28.002599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:28.093943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:28.173994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:28.178063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300d9229-f1db-40c0-9d66-4ad3b98dd6a8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:28.216900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:28.218565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:28.218841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:28.219110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:28.236571 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7238d078-5a37-4913-a35b-4bdfcd2d1c34 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:28.244547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:28.251720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:28.252574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:28.253339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:28.260622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 267/542] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:28.266879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:28.306282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:28.307416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:28.307693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:28.307961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:28.308653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:28.308959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:28.310417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:28.336172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 276/543] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:28.342118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:28.465079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9567c521-2209-41dd-a543-c342666221b1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:28.487857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa14fa2-b989-408b-8296-f72c5e960076 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:28.498953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:28.528580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:28.529196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:28.529474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:28.529773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:28.530502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=20154db5efc6482bbae149b81fb06cd2) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:28.530806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:28.532234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:28.535282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:28.536352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:28.537251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:28.537835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:28.540852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:28.540852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:28.540852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:28.540852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:28.542050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 277/544] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:28 2022] GET /identity/v3/role_assignments?user.id=20154db5efc6482bbae149b81fb06cd2&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:28.552557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 268/545] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:28.558802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:28.714882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ea477-4402-4c26-8e17-95f62eb307d7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:28.749976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:28.750532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:28.750811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:28.751081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:28.762483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:28.763927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:28.778616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:28.779498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:28.780294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:28.780294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:28.780294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:28.780294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 278/546] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:28.785659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:28.921957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f38a6a-586d-4a21-b5fe-eacb92db327f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:28.979453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:28.980070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:28.980349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:28.980617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:28.981265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:28.981584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:28.983653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:29.001047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 269/547] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:29.006424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:29.066171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:29.126961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a48ef-1531-4ccf-a539-b483b4ad03ee None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:29.169172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:29.169836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.170117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.170385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.178282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:29.179727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:29.195205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:29.196130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:29.198321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 279/548] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:29.202299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:29.256144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:29.319712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a395c4d-91b4-4e68-b7f9-325157921c1d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:29.360927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:29.361543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.361844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.362119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.362846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:29.363155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:29.364581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649decf-2693-48e8-9d9f-1042f882d27f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:29.382635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 270/549] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 05:33:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:29.388212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:29.502222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df27025-59ef-4b66-8911-1b8e5f986f84 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:29.516396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:29.544716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca791f20-7e28-4412-aa3d-3cfa1448126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:29.545332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca791f20-7e28-4412-aa3d-3cfa1448126d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.545630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca791f20-7e28-4412-aa3d-3cfa1448126d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.545906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca791f20-7e28-4412-aa3d-3cfa1448126d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.594644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb1e90-9ad7-45df-b4bf-19f9dd313b29 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.595035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb1e90-9ad7-45df-b4bf-19f9dd313b29 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.595228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb1e90-9ad7-45df-b4bf-19f9dd313b29 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.596819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 271/550] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:29.619106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e968ec3-3cc0-4e5d-af69-b7d1533da193 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.619412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e968ec3-3cc0-4e5d-af69-b7d1533da193 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.619665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e968ec3-3cc0-4e5d-af69-b7d1533da193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.657652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca791f20-7e28-4412-aa3d-3cfa1448126d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:29.659110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca791f20-7e28-4412-aa3d-3cfa1448126d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:29.674696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6e968ec3-3cc0-4e5d-af69-b7d1533da193 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:29.675905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6e968ec3-3cc0-4e5d-af69-b7d1533da193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:29.809160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 272/551] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:29.821584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317659f-bf3a-49b4-9168-8b38700b91d6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.821584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317659f-bf3a-49b4-9168-8b38700b91d6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.821584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317659f-bf3a-49b4-9168-8b38700b91d6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.825977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 273/552] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:29 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:29.826203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 280/553] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:29.833692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317659f-bf3a-49b4-9168-8b38700b91d6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:29.835363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:29.835741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:29.836131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:29.869589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:29.885587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:29.885587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:29.904161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:29.980331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317659f-bf3a-49b4-9168-8b38700b91d6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.048327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.049372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.049912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.050385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.065272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:30.067175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:30.108004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:30.109113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:30.110354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:30.110354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:30.110354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:30.112451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 274/554] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:30.120403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.122622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 281/555] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:30.125025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.293593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a986ccd8-de24-41e7-a26d-d824a5272d45 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.339413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:30.347544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cb1351-fb17-421e-93cf-0b00f6165bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.348664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb1351-fb17-421e-93cf-0b00f6165bff None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.348937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb1351-fb17-421e-93cf-0b00f6165bff None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.349716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cb1351-fb17-421e-93cf-0b00f6165bff None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.376494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cb1351-fb17-421e-93cf-0b00f6165bff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-f7cb1351-fb17-421e-93cf-0b00f6165bff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:30.386726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:30.387679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:30.388580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 282/556] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:30.389775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0b3fe-fb24-45be-8357-63c9e1926c23 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.393251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df86f02-3f85-4ed0-88da-d0d2f6165051 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.394595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0b3fe-fb24-45be-8357-63c9e1926c23 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.394906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0b3fe-fb24-45be-8357-63c9e1926c23 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.400678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 283/557] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:30.400678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0b3fe-fb24-45be-8357-63c9e1926c23 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.401812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:30.435439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.437265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.437791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.438241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.439054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:30.439532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:30.441609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c203664-57d1-46cc-a1e1-ab66436ecf71 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:30.459317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 275/558] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:30.461486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.461972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.463209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.484743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:30.485963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:30.505161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0b3fe-fb24-45be-8357-63c9e1926c23 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.541438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.542064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.542341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.542611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.543265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:30.543565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:30.544921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:30.552964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 284/559] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:30.553782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.563633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 276/560] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:30.565168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.629258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:30.653555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:30.677551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce956a68-8dfd-48a3-9145-b14f06f76fe2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.698929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9992bf-a621-4743-92d7-f556ed7b4218 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.713908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.714554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.714835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.715109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.721977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:30.723538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:30.736336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:30.737222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:30.738094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:30.738507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.738507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.738507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.738507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.741911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 285/561] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:30.742506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.754079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:30.759319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:30.760187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:30.760992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 277/562] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:30.762292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.847624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b5540-910f-4e7d-a6e0-af44553f44ea None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.870544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f4a2d-2ce3-4faf-a19e-a71368ba399a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:30.873993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:30.887076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.887626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.887902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.888177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.899298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:30.901504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:30.905540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:30.912533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:30.913142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:30.913423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:30.913741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:30.914439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:30.914748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:30.916746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:30.917504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:30.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:30.919265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:30.919265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:30.919265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:30.923163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 286/563] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:30.923914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:30.936874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 278/564] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:30.937598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.032106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff606fb-b8b7-4401-bea9-d6c07fafc088 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.041780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992fdb1-f1af-45a5-9a34-c7a3c17a4a43 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.066949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.067561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.067848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.068120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.068777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.069081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.070487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.075063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.075687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.075964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.076233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.078360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 287/565] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.079148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.086379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:31.091132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:31.092219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 279/566] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:31.093276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.122996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.144088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.183469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdde96-27ff-4b6b-b21c-a7a839776172 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.193805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab72afb-3b1f-415d-b6d2-1b5c9679e489 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.217159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.217841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.218130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.218395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.219023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.219317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.220669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aba7cb-1a27-46f1-bc20-f9559025720a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.226834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.227417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.227692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.227961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.229442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cdca2e846d9340a5ab6ee8356115a6c7) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.229775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.231174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.232654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 288/567] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.234467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c414-153b-4d47-9dda-6628255a190a None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.234759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c414-153b-4d47-9dda-6628255a190a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.235191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080c414-153b-4d47-9dda-6628255a190a None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.236673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 289/568] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:31.237201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080c414-153b-4d47-9dda-6628255a190a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.240474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 280/569] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/role_assignments?user.id=cdca2e846d9340a5ab6ee8356115a6c7&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.241293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.356556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.363125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080c414-153b-4d47-9dda-6628255a190a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.366989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c69f9-e24b-4d02-90ea-618ba1c4ffef None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.383389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.402845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.403761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.404405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.404858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.405434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.406092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.406368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.406633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.407288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.407591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.409019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.417578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 281/570] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.417804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.418450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.419657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-533984d7-c9ad-4a3f-9704-9f1fa7afe6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:31.436753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:31.437924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:31.439019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:31.439019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:31.439019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:31.445066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 290/571] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:31.446621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.447078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.447491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.466807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:31.468454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:31.524053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d16b1-c060-40e0-8cf8-b521cd2f526d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.554602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 291/572] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:31.554602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.571486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.572320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.572954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] PATH_INFO: `/v3/users/glance` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.582682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.584267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:31.596698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:31.597684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:33:31.598619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:31.602527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 282/573] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:31.604261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.646479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.670514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.699405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640418a8-e8f0-427c-a5c8-6cfe3a14dabb None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.725608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eeca61-5e7e-4060-a20b-c1a309ec513c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.738091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.739099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.739604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.740067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.751040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.752866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.766844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0497562-790d-4458-af16-251678da8881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.767455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0497562-790d-4458-af16-251678da8881 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.767733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0497562-790d-4458-af16-251678da8881 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.768003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0497562-790d-4458-af16-251678da8881 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.768639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0497562-790d-4458-af16-251678da8881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.768944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0497562-790d-4458-af16-251678da8881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:31.770147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:31.771036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:31.771932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:31.771932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:31.771932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:31.772096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0497562-790d-4458-af16-251678da8881 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.774063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 292/574] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:31.775650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.789283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 283/575] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.790230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0497562-790d-4458-af16-251678da8881 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.886146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e549026-5643-4887-92de-b4d0e4c57849 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.897336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.904854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0497562-790d-4458-af16-251678da8881 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:31.917145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:31.923163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.923714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.923994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.924263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.924987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.925293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.926759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.941360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:31.942223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:31.942504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:31.942777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:31.943387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:31.943683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:31.949981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 293/576] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.950133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:31.950214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:31.965854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 284/577] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:31.965854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.057958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a50a7-bbad-4ff1-ac61-14e1a73da767 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.067808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f427bea-ee3d-4b15-8c5b-03c55195d7ec None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.099534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.100252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.100528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.100797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.102174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.103916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.104865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.105208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.109051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.110528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.115724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:32.122013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:32.122924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:32.123736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:32.129433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:32.130335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:32.131244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:32.131750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 285/578] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:32.131818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 294/579] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:33:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:32.131818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.131936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.152727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.179530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.235060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708533-09cc-46c4-8d17-a6112ac65617 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.245978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863815e8-2506-49d9-b492-6139c2c077b2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.274177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.274917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.275299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.275667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.284098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.285592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.288419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.289050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.289328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.289627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:32.298216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:32.299389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:33:32.300443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:32.302018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.308258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.308470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 295/580] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:32.308470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:32.326190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:32.327339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:32.328324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:32.328324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:32.328324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:32.328324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 286/581] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:32.328876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.400630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.423213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf0b31-4ddc-450e-bb00-a181ef62be88 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.434394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.440712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08920403-9d87-42fc-8455-df5f2d7632cf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.458129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.458734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.459013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.459286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.459944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:32.460245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.461617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.474790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 296/582] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:32.475428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.477608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.478408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.479103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.479437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.480233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:32.480610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.482169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.500096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 287/583] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:32.504645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.591831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60d8f28-cafd-4430-a933-6dde08535e34 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.610575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3078f49-42b5-4b18-8bb6-289e9457050a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.627019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.627629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.628000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.628274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.628992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:32.629296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.630766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.637939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.647502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.648065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.648342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.648611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.649255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:32.654727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.654864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 297/584] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:32.654864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.655028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.666842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 288/585] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:32.667594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.690744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.765670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc2a2c-3934-4afa-a6f6-5400d3bd965b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.781124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066706d2-79f6-4631-a63d-bfe1163df963 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.800210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.800848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.801128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.801400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.811490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.812931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.819096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.819971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.820353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.820720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:32.828285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:32.829398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:32.834940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:32.834940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:32.834940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:32.834940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 298/586] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:32.834940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.840606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.842224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:32.857251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:32.858508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:32.859565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:32.859565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:32.859565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:32.860033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 289/587] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:32.860823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:32.901864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.923986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:32.944897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab26742-77ae-4761-a5c4-c703d57003e5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.968954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864f6bed-fe62-4a6e-bef6-18e31e73c38a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:32.981879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:32.982504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:32.982780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:32.983049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:32.993532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:32.995495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.006514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.007145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.007428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.007700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:33.012363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:33.013435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:33.014519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:33.014519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:33.014519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:33.020537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 299/588] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:33.020667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.020781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.020872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:33.032610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:33.033717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 290/589] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:33.039390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.129943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d94582-e1ae-4989-9b76-e90b86eb6cfe None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.145173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.149385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e0b55-b0aa-444d-91ae-0665f309cf93 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.164872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.165496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.166321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.166991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.167396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:33.167696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.169061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.170563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.181840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 300/590] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:33.182644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.186578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.187190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.187473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.187748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.188399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:33.188707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.190128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.202232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 291/591] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:33.202837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.289747 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f233dd7-dad9-4be2-aad1-fbe4e0570c60 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.308680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af92cf-f2c3-4843-8e5f-d9715af7ce8b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.323813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.324415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.324691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.324960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.325616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:33.325916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.327275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dbeb7d-d15f-4973-a8a2-5222d4f970e2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.338720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 301/592] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:33.340140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe0c8e-8fca-4947-82e5-011dca29a100 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.340621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe0c8e-8fca-4947-82e5-011dca29a100 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.340992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe0c8e-8fca-4947-82e5-011dca29a100 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.345599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 302/593] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:33.345728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.345838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe0c8e-8fca-4947-82e5-011dca29a100 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.345925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.345925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.345925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.345925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:33.345925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.347161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.359394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 292/594] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:33.360327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.378816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.409170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.454844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe0c8e-8fca-4947-82e5-011dca29a100 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.463430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fca0fe-37cb-43f0-938e-214d6bb72d30 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.488339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.489047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.489334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.489635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.496058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.496500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.496680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.496958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.497229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.498016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.503741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.505209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:33.511205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:33.512132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 303/595] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:33.513029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-afc390a4-ea8e-4e4a-98fa-a312a82edbdc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:33.522433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:33.523410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:33.524280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 293/596] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:33.526290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.526514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.526694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.544503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:33.545785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:33.612157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 294/597] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:33.612925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886115f7-9446-48ff-8e8e-e68bb436fda6 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.613035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.627231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.631006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.652858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.653595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.653881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.654149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.661916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.663764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:33.676444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:33.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:33.678614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:33.683107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 304/598] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:33.683730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.738396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906521d-3c3c-4745-847b-ca2f9085c17e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.774245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.775078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.775454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.775826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/960468d880f34ea2a267b6ada5fb30bd/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.791467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=960468d880f34ea2a267b6ada5fb30bd, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.792964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.793100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0787e9-0986-48fe-9e5f-811e84ff5120 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.807028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 295/599] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:33 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/960468d880f34ea2a267b6ada5fb30bd/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:33.807590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.839858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.840476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.840753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.841081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.841884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:33.842226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.843868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:33.873421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 305/600] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:33.873421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:33.901425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.916627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:33.955703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491efaa-1843-4bb7-bd96-37b9c8076f1c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:33.993181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:33.994281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:33.994564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:33.994883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:33.995886 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:33.996187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:33.997657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.005046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc46942-81c6-4ef9-9519-aa821c74dc29 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.010889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 296/601] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:34.015688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.046079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.046858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.047238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.047610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.048374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:34.048750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.050280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.062539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 306/602] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:34.063357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.119938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47a0a8-c71c-486b-8c32-526f2c2835ee None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.144590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:34.163565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.164211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.164483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.164753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.175290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:34.178425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.180556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.186980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc4c4b-d984-4724-a32b-b4733564707b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:34.206580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:34.211880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:34.213442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:34.213442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:34.213442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:34.213442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 297/603] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:34.213442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.237416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.238211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.238667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.238947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.253848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:34.258862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:34.259899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 307/604] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:34.261350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.335234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826003f-22fc-46bf-89ba-58b6146490de None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.378496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.379218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.379497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.379765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/4e926d4a438e42ecaeab44f711e0b3c9/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.389481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3ba1a-2fc5-4920-80c0-8ca05ba9fb6f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.396363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=4e926d4a438e42ecaeab44f711e0b3c9, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.397843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.421695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 298/605] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:34 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/4e926d4a438e42ecaeab44f711e0b3c9/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:34.421695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.438999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.439803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.440155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.440504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/1c7f12a5dc6446eaa94a96dbb44ccdaf/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.445954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:34.460833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=1c7f12a5dc6446eaa94a96dbb44ccdaf, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.462642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.486059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 308/606] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:34 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/1c7f12a5dc6446eaa94a96dbb44ccdaf/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:34.488119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.497198 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:34.563056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e66799d-5f4b-48af-8070-58b88753950e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.622205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.623651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.624086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.624501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.625490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:34.626099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.627727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.647846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 299/607] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:34.656490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.694659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2fcb63-a07b-4af3-9787-020a06d393ef None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.740194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.740849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.741084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.741410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.742268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:34.742574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.744579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.760005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 309/608] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:34.766814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:34.777706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:34.808628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e573868c-ecc2-4833-ae14-4bc2eeec39d2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.851357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:34.870828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:34.871756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:34.872192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:34.872601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:34.884046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:34.887567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:34.916143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:34.917252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:34.918430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:34.920978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 300/609] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:34.931815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98665cb9-c4df-4ca8-a3bc-b57ccba65234 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:34.947282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:35.001475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.004698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.011812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.012719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.026946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.028918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:35.045929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:35.047045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:35.048103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:35.048103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:35.048103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:35.049944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 310/610] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:35.058553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:35.132475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04837b2-8d76-41ae-bde7-797210bc0d9f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:35.184074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.184074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.184074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.184074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.184522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:35.184522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.184522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.187125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb0364-2c45-422d-a618-45cb4ad2f797 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:35.201858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:35.207165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 301/611] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:35.214592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:35.221032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:35.271033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.271771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.272161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.272526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.273272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:35.273706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.275281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.312088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 311/612] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:35.317193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:35.348865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30740e8a-504c-48af-81fd-1d7c5ac54acd None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:35.411784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.412690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.413078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.413539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/20154db5efc6482bbae149b81fb06cd2/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.431614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c1f63d-fc3a-48a9-ad00-e09aea08a584 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:35.443367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=20154db5efc6482bbae149b81fb06cd2, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.444953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.469008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.469671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.470016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.470290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.479749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.481197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.481500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 302/613] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:35 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/20154db5efc6482bbae149b81fb06cd2/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:35.494191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:35.495351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:35.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 312/614] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:35.501490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:35.602746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:35.675564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da695a-4c28-46d7-a436-dcbc027d2f13 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:35.712870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.713857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.714299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.714766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.715646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:35.716099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.717698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.737657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 303/615] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 05:33:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:35.743327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:35.780728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:35.850657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb643ca-1081-4045-8c12-787314fc9b41 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:35.887302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:35.888004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:35.888284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:35.888600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:35.900155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:35.901440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-50e5ec46-2664-40dd-9b50-5b97e6d2e0ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:35.917387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:35.918406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:35.919268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:35.919268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:35.919268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:35.919268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 313/616] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:35.924621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:36.033410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740b97f-b79f-4380-a0e6-e197e8b948f6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:36.033755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dfb6c-553a-4b8f-aaf0-2ecb78195a74 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.034264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dfb6c-553a-4b8f-aaf0-2ecb78195a74 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.034264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9dfb6c-553a-4b8f-aaf0-2ecb78195a74 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.036830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 314/617] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:36.041069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.041350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.041656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.044763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:36.062343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:36.063760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:36.075515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:36.076126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.076401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.076985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.077925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:36.078231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:36.079780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:36.094903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 304/618] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:36.102678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:36.180183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 315/619] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:36.289381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b38a0f-bcbf-47fd-b3b1-356aa5b7d299 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:36.339969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:36.340964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.341448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.341973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.354915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:36.356718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:36.371594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:36.372647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:36.373499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:36.375616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 305/620] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:36.381430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:36.404612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:36.501348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90fbcf-3901-4666-b99f-738bacf6fa71 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:36.541493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:36.542121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.542476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.542912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.543717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:36.544029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:36.545478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:36.557837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 316/621] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:36.566677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:36.583626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:36.615562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:36.715912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:36.748638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4804d64f-f095-4748-b06f-593959c2267a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:36.758225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30654bd9-8c79-4f8c-a104-62b8963e10a8 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:36.809817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c08a8-aca1-4186-b103-70c6eae10980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:36.809817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c08a8-aca1-4186-b103-70c6eae10980 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.809817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c08a8-aca1-4186-b103-70c6eae10980 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.809817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c08a8-aca1-4186-b103-70c6eae10980 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/cdca2e846d9340a5ab6ee8356115a6c7/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.811876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60adcd2f-73e0-4dcb-b3ee-088e00b0e277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:36.812516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adcd2f-73e0-4dcb-b3ee-088e00b0e277 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.812790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adcd2f-73e0-4dcb-b3ee-088e00b0e277 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.813056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60adcd2f-73e0-4dcb-b3ee-088e00b0e277 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.827944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c08a8-aca1-4186-b103-70c6eae10980 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=cdca2e846d9340a5ab6ee8356115a6c7, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:36.829402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c08a8-aca1-4186-b103-70c6eae10980 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:36.830028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60adcd2f-73e0-4dcb-b3ee-088e00b0e277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-60adcd2f-73e0-4dcb-b3ee-088e00b0e277 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:36.838459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:36.839480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:36.840295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:36.840908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 317/622] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:36.846456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659a19d-3f91-4ef1-b486-8acda2ccfa8e None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.846821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659a19d-3f91-4ef1-b486-8acda2ccfa8e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.847142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659a19d-3f91-4ef1-b486-8acda2ccfa8e None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.853026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 318/623] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:36.853026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659a19d-3f91-4ef1-b486-8acda2ccfa8e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:36.858100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 306/624] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:36 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/cdca2e846d9340a5ab6ee8356115a6c7/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:36.859803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:36.860561 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:36.861059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:36.882298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:36.883791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:36.960065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 307/625] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:36.970685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659a19d-3f91-4ef1-b486-8acda2ccfa8e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.008951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.010256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.010663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.011172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.012078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:37.012551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:37.014108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1692fc84-7d73-41e2-8b0c-e85d0e3b5b64 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:37.020247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:37.024307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 319/626] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:37.029946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.047820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:37.096225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9948c9-0561-49e4-a977-c852964d6f87 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.096751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9948c9-0561-49e4-a977-c852964d6f87 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.096751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9948c9-0561-49e4-a977-c852964d6f87 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.098376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 320/627] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:33:37.104896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.105306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.105717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.127948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:37.129143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:37.194957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5a8dd7-4370-4b6b-89ec-509fe1cc6b7e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.254062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.255311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.256553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 321/628] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:37.257047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.257552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.289368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:37.291752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:37.313034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:37.314083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:37.314985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:37.314985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:37.314985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:37.316742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 308/629] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:37.322850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.336185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.452185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:37.499782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf52cd7-bc15-49de-92a7-373e4793ba79 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.535846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6afa484-3fa4-4b98-965f-8c3477c7a50c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.543654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.547168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.548336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.549604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.569201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:37.575384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:37.576337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:37.577264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:37.581280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 309/630] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:37.582177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.590288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:37.597336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.598349 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.598911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.599413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.600270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:37.600776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:37.602679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:37.617362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 322/631] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:37.619500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.688044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec03735-5b6a-49d2-9e11-f4458e479882 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.708551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:37.725223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.725879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.726155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.726425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.738011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b9d29b-5818-4513-b514-2a5f1c30f61e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.738816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:37.744733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:37.745707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:37.746543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 310/632] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:37.747200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.779986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.781067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.781611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.782093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.782948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:37.783534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:37.785286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:37.795070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 323/633] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:37.796321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.852808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5942a9c-ac67-41d0-a93f-995233cbdb50 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.854639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:37.890356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.891362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.891891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.892304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] PATH_INFO: `/v3/users/glance` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.901248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:37.903321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:37.908810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f07ec2-6206-4e94-ac55-66ca1ef18901 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:37.915611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:37.916776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 05:33:37.918070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:37.924631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 311/634] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:37.925784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.947982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:37.948748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:37.949122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:37.949553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:37.950637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:37.951036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:37.952538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:37.962359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 324/635] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:37.964111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:37.972326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.035191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eb6ed8-7757-45da-8e71-2800774a1d6d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.068894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.069521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.069836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.070106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.071147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d18eac-1426-4ada-b07c-043957271d6e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.081652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.083107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.085935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:38.099253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:38.100231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:38.101137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:38.101137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:38.101137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:38.101137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 312/636] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:38.101137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.115391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799183e-ff10-40e6-972a-356749196410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.116064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799183e-ff10-40e6-972a-356749196410 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.116274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799183e-ff10-40e6-972a-356749196410 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.116548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799183e-ff10-40e6-972a-356749196410 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.117271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799183e-ff10-40e6-972a-356749196410 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:38.117594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799183e-ff10-40e6-972a-356749196410 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.119775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799183e-ff10-40e6-972a-356749196410 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.143431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 325/637] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 05:33:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:38.144499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6a40-b3be-44af-a22d-d963bf785b50 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.144782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6a40-b3be-44af-a22d-d963bf785b50 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.145050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e6a40-b3be-44af-a22d-d963bf785b50 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.149611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 326/638] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:38.149611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6a40-b3be-44af-a22d-d963bf785b50 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.208061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.219274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e05690-4dd3-4c84-83bf-0b267a00f638 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.257909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e6a40-b3be-44af-a22d-d963bf785b50 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.261730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.262469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.262852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.263277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.277089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.278690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:38.293503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:38.295160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:38.296233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:38.296233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:38.296233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:38.297984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.298690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.299062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.299421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.300146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:38.300491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 313/639] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:38.300593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.301326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.302138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d24605-409e-49ab-b406-9eb257ae3aa2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.315810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 327/640] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:38.317052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.317497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.317882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.336101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:38.337840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:38.365380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.410565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 328/641] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:38.411637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.427844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c5723-217e-4d5c-b80c-b64d87ea5779 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.466482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.467338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.467804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.468233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.485696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.549424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.551363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.551827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da799c04-9499-4d28-904f-1a0fda82d8f7 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.598897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.600025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.600383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.600894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.601716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:38.602100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.604602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.621748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 329/642] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:38.621748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.656812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.675106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 314/643] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:38.675975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.724861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.754788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a828b-6964-4803-b108-3cf47ede730d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.785952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7326ae-9ef9-4a78-b367-76927a1c4c91 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.788239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.788909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.789146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.789414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.797961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.799384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:38.810958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:38.811946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:38.812875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 330/644] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:38.813462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.827871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.828513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.828784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.829062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.836310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.837817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:38.849421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:38.850564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:33:38.851851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:38.856206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 315/645] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:38.857069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:38.906087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.925790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0039f350-fe10-479d-8933-7c465d1ecb1e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.960522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:38.961306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:38.961724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:38.961838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bf98b-4b66-41fa-9e38-fbca05ef839e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:38.962697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:38.966134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:38.974006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:38.975450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:38.990681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:38.991807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:38.992861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:38.992861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:38.992861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:38.996316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 331/646] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:38.997144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.002523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.003156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.003437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.003708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.014023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:39.019161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:39.020229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 316/647] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:39.021075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.099113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10559e-fe21-4f5d-9beb-3ef951b9fbbe None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.120637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596807d-1c1c-4c89-a634-0bbef50059ba None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.132886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.133491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.133814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.134118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.135145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:39.135561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.137106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21dcbe-82d8-439f-869c-a78d2b4ca2f7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.138763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.154237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.154841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.155122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.155392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.156055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 332/648] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:33:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:39.156179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:39.156411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.157336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecab7b-e67d-47aa-b8e0-6010a0ec17f1 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.157652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecab7b-e67d-47aa-b8e0-6010a0ec17f1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.157890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.157985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecab7b-e67d-47aa-b8e0-6010a0ec17f1 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.159099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 333/649] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:39.163402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecab7b-e67d-47aa-b8e0-6010a0ec17f1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.177684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 317/650] 10.222.0.73 () {56 vars in 1137 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:39.178370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.201538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.282137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0237b2-cd6e-4192-be9d-7d6ca330047a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.288070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecab7b-e67d-47aa-b8e0-6010a0ec17f1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.316429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.317024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.317302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.317589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.318245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:39.318544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.319908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.322139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.322748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.323018 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.323287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.323912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:39.324208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.325550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0423b6ad-c0b5-4c2d-8040-7b471fce7523 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.328393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 318/651] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:39.329182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.336587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 334/652] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:39.338341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.338619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.338886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.357141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:39.358368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:39.404567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.413769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.424781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 335/653] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:39.425704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.441005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71bcd64-6050-409c-8963-7b853334d76a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.478418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.479580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.480036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.480500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.530113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801fa3e9-404f-45b8-9fcb-fc10d009491d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.566700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.567313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.567593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.567862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.569084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.571394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.617997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.637741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.639204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.662690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.682075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 319/654] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:39.682994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.742633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 336/655] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:39.743981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.785294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16129a62-acaa-428f-ab37-004b5f5b3767 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.819395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.820016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.820294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.820565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.821542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.831468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.832946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.845760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8bc0a-52cb-41a1-abe7-84fba9976e64 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:39.847333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:39.848471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:39.849328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:39.849328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:39.849328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:39.849328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 320/656] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:39.849328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.886129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:39.888232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.888868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.889145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.889414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:39.899412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:39.900998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:39.912291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:39.913194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:39.914330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 337/657] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:39.914815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:39.957309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b8ad2a-dfb8-4f27-92c2-b710529ef377 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:39.990756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:39.991380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:39.991662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:39.991938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.003531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.004980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.011676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f74e15-19fa-4be0-9af5-1187a2cb4c0b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:40.019876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:40.020995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:40.022167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:40.022167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:40.022167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:40.022167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 321/658] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:40.022167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.050177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.050724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.050998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.051264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.062019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:40.067130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:40.068098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 338/659] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:40.068940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.073396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.111520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.125201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1759-d6c4-4c83-a811-29bdb7df58d8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.157933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.158555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.158832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.159102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.169297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611758b-3f14-4fc0-b37d-cf375b0fdd3f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.170421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.171873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:40.187020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:40.188130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:40.189232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:40.189232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:40.189232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:40.189721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 322/660] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:40.190554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.212368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.212980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.213250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.213516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.214194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:40.214491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.215892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.229403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 339/661] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:40.230351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.303237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb83653-6ed1-4a6f-b017-06c91e8403cc None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.304900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.336962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.337635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.337933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.338226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.338938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:40.339252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.340697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.342859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b04f2-c552-4cc0-9a4e-8d3d1c1a359d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.349555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.354708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 323/662] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:40.355670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.377942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.378542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.378818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.379086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.379716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:40.380013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.381374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.392173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 340/663] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:40.392954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.453677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e17d4c-68ce-43e0-a42a-0e4034c3108b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.487587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.488176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.488455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.488718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.489366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:40.489687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.491071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.491810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac19102-eacb-4618-b68f-e3e42122f1e5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.498930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 324/664] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:40.499709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.518347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.531159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.531762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.532041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.532308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.532946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:40.533247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.534638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.546876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 341/665] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:40.547679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.565522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.605535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d616f-8303-4030-b2cc-31e114e49609 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.638635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.639253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.639527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.639790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.646336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fdfd7-0aab-4daa-bf30-086637dd07cf None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.646964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.648389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:40.659915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:40.661030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 325/666] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:40.662046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.685244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.685877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.686161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.686434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.687160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=960468d880f34ea2a267b6ada5fb30bd) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:40.687468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.688914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.694539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 342/667] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/role_assignments?user.id=960468d880f34ea2a267b6ada5fb30bd&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:40.695297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.761327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.772833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3b6a2-612d-4737-a7c6-af6f1b850cfe None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.798836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:40.811909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.812584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.812864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.813137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.820701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.822737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.827109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eb28a-29ec-441b-b8e9-fe9014eb9881 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:40.835216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:40.836135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 326/668] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:40.838003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.871778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:40.872414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:40.872689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:40.873600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:40.886453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:40.887894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:40.902824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:40.903794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:40.904879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:40.904879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:40.904879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:40.904879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 343/669] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:40.905115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:40.966190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7e612-e2a0-4150-a11a-688b1ce10ec7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.001738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.002381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.002664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.002938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.010004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.011447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.013377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.017729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef820297-4b9b-4157-8a43-7893a2215c70 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:41.023381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:41.024243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:41.025150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:41.029136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 327/670] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:41.029985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.058713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.059335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.059876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.060151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.061338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:41.061943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.064084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.076187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.085625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 344/671] 10.222.0.73 () {56 vars in 1139 bytes} [Wed Aug 24 05:33:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:41.090529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.141468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f63a9-2497-4e5e-9378-a4fd1d8daf10 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.178589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.179193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.179467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.179737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.180416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:41.180715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.182764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.199292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 328/672] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:41.199292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.199557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50278a5e-f9ec-4c94-8eaf-8ea3f77a709c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.236969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.237581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.237892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.238163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.238805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:41.239106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.240469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.256796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 345/673] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:41.256796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.277061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.321201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb238a-7f13-4776-8a9a-4061010d1c03 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.322892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.358238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.358883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.359024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.359291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.359970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:41.360272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.361809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.365625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fe97db-a041-407e-9656-2f87eae982ef None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.374141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 329/674] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:41.374789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.406762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.407376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.407654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.407924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.479203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.480705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.482924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bbe38-5417-438a-971c-e852c0706737 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.498621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.518241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.518841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.519116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.519403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.520121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=4e926d4a438e42ecaeab44f711e0b3c9) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:41.520423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.521885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.528222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 330/675] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/role_assignments?user.id=4e926d4a438e42ecaeab44f711e0b3c9&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:41.529034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.567975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.619705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 346/676] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:41.619705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.661090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c343ae5-5f58-48f9-828c-d7889fa11dc6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.701730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.702521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.702911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.703279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.711560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.713046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:41.725726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:41.726698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 331/677] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:41.727636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.756203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0655899-11d2-403f-b655-0b070c01750b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.777897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.806092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.806796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.807253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.807527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.808272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=1c7f12a5dc6446eaa94a96dbb44ccdaf) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:41.808576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.810528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 347/678] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/role_assignments?user.id=1c7f12a5dc6446eaa94a96dbb44ccdaf&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:41.818402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:41.841947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:41.862920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06e6c0-d664-42bc-9c8a-9d16d83cf114 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:41.920597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:41.921159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:41.921434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:41.921732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:41.950393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:41.951881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:41.976840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:41.978523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:41.980060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:41.980060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:41.980060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:41.986463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 332/679] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:41.992152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.000553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa291f-4706-45d2-b4e1-669f435323cc None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.060323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.060530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.060778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.061009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.061769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:42.062073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.064485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.085777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 348/680] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:42.092246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.121420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b589825-ec80-47cb-bb0f-2794eddf04f8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.135836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:42.162938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.163540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.163811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.164076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.164713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:42.165010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.166618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.180327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 333/681] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:42.182053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:42.194041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.217132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab2d855-4793-4557-914f-f7ed7250d83a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.254788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.255421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.255700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.258768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.339658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.341126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.397593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef152598-4a29-4b88-ac14-27517e4e2984 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.412788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:42.461699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.462870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.463270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.463420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.479237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.480753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:42.496077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:42.497181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:42.498294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 334/682] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:42.504611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.533444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 349/683] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:42.538981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.577070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:42.661846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917e99f6-8e30-4c57-9a11-ef16a7acfe27 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.677621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09895ce9-157f-40ca-8f22-82831ed9ab91 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.725294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.725855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.726134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.726404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.727099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:42.727401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.729755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.731677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.732310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.732583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.732849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.741601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:42.753633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.755129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.761604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 335/684] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:42.766289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-a41c88ab-9971-424c-8256-c1fb41c6a7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:42.786050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:42.787008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:42.787861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:42.787861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:42.787861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:42.788428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 350/685] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:42.789382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64faa1-7c0a-4dc9-b702-2bccc3631702 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.789693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64faa1-7c0a-4dc9-b702-2bccc3631702 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.789967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b64faa1-7c0a-4dc9-b702-2bccc3631702 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.793656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 351/686] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:42.794434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64faa1-7c0a-4dc9-b702-2bccc3631702 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:42.879084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449730b-bc75-4c54-97ce-85398be2458b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.898458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:42.917609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.918535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.919021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.919410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.920228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=20154db5efc6482bbae149b81fb06cd2) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:42.920634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.922191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9cbc2-a97d-42f7-b44a-6983dea93f94 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:42.928412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 336/687] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:42 2022] GET /identity/v3/role_assignments?user.id=20154db5efc6482bbae149b81fb06cd2&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:42.929685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.930062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.930425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.931978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b64faa1-7c0a-4dc9-b702-2bccc3631702 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:42.950969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:42.952276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:42.983293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:42.984399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:42.984893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:42.985441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:42.986438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:42.986965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:42.988735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:43.004650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 352/688] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:43.010552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:43.054960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 337/689] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:43.134788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:43.191883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d6f1e-9dfd-4d21-b54e-50f0d7611b4d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:43.226368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:43.228314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:43.229104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:43.229597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:43.237078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:43.238802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:43.251458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:43.252347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:43.253308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:43.255740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 353/690] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:43.266999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:43.451852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b5be55-b2e0-4864-93df-cda1d9d15845 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:43.480728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:43.519971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:43.521265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:43.521998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:43.522672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:43.524093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:43.525341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:43.527854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:43.553677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 338/691] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:43.558755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:43.622923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:43.738881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ecead5-f294-42a2-9e79-e047aceb7611 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:43.739738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd4672-13a1-470b-83fa-4086f36c9fa5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:43.764028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:43.792580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:43.793604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:43.794113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:43.794590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:43.811254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:43.813509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:43.816452 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:43.817334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:43.817859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:43.818336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:43.819266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cdca2e846d9340a5ab6ee8356115a6c7) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:43.819786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:43.821952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:43.838343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 354/692] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:43 2022] GET /identity/v3/role_assignments?user.id=cdca2e846d9340a5ab6ee8356115a6c7&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-729fccbf-5719-4cab-814d-c0e0a05e6e9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:43.843753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:43.844808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:43.845734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:43.845734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:43.845734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:43.847979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 339/693] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:43.854285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:43.933656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210a5b-2edb-422c-8e9c-079b0e8e5132 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:43.934293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210a5b-2edb-422c-8e9c-079b0e8e5132 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:43.934769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e210a5b-2edb-422c-8e9c-079b0e8e5132 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:43.936651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 340/694] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:43.951029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:43.951652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:43.952152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:43.995200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:43.996837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:43.997920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66a3a8-ff37-4a2a-b20c-a3d82d8a489e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:44.030483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:44.040368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:44.042240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:44.042539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:44.042891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:44.043449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:44.043752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:44.045245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:44.063585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 355/695] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:44.065275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:44.088678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 341/696] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:44.106764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:44.174411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c5cbc-fa8e-4f1c-814e-1fa3be95ed21 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:44.210605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ea1ce-a2d5-4034-82b8-5480c28a5a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:44.211594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea1ce-a2d5-4034-82b8-5480c28a5a17 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:44.212640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea1ce-a2d5-4034-82b8-5480c28a5a17 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:44.213142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ea1ce-a2d5-4034-82b8-5480c28a5a17 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:44.214016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ea1ce-a2d5-4034-82b8-5480c28a5a17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:44.215675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ea1ce-a2d5-4034-82b8-5480c28a5a17 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:44.240508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 356/697] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:33:44 2022] POST /identity/v3/users => generated 254 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:33:44.395514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:44.463979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991794-8603-4601-ac5b-5c94cbaba9ea None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:44.464194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991794-8603-4601-ac5b-5c94cbaba9ea None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:44.464440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22991794-8603-4601-ac5b-5c94cbaba9ea None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:44.466880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 357/698] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:33:44.472107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:44.472380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:44.472644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:44.496980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:44.498248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:44.516427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffab14cb-3b2e-4094-93a2-e4cda17b4c82 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:44.567457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:44.568135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:44.568419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:44.568689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:44.584281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:44.586461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 358/699] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:44.590672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:44.591620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:44.592473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:44.597634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 342/700] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:44.598912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:44.626996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:44.714296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7bc93-684d-4a96-872a-8447a6febb48 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:44.760088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:44.760710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:44.760993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:44.761383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:44.762039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:44.762347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:44.763740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:44.777972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 359/701] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:44.783806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:44.812419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:44.862574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:44.950812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd8f48-3471-4c92-b08f-a8e1e6ad9689 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.024740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.025289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.025703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.025877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.026612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:45.051020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.052490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.061890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ef1b6-46d4-492f-85bd-671ffe195ab3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:45.082290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:45.083298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:45.084186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:45.084186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:45.084186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:45.094772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 343/702] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:45.102952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:45.136783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.137430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.137733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.138004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.138783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.141066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:45.157687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:45.158761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:45.159615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 360/703] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 05:33:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:45.170972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:45.202315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:45.283139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0c457-5ac6-449e-a554-ce85fe818c58 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.354736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.355354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.355630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.355899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.356545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:45.356843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.358451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.373771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 344/704] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:45.380352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:45.382689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09100c93-e900-4d94-a258-e0d6426b0cdb None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.457281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.457932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.458210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.458549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.459195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:45.459499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.460882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.472336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 361/705] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 05:33:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:45.476908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:45.500521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:45.557646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9dd6d-d940-4d38-ac18-058f71907e93 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.585917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:45.597534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.598455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.598928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.599366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.607525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.609304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.614548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e355c-67f9-4b8b-b0ad-39745e6d9f51 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-ad764d63-dc49-4dea-8f8b-d56bd182a147 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:45.622150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:45.623281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:45.624341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:45.626469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 345/706] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:33:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:45.627837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758b847-044b-41b0-a784-d4b34838c550 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.628247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758b847-044b-41b0-a784-d4b34838c550 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.628647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758b847-044b-41b0-a784-d4b34838c550 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.630285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 346/707] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:45.631758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758b847-044b-41b0-a784-d4b34838c550 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:45.656001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.656610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.656890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.657163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.657820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:45.658182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.659624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cebdfe3-96d5-4b7a-8189-53c13706bdf3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.673590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 362/708] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 05:33:45 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:45.673590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.673590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.673590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.689896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:45.691184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:45.770377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 363/709] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:45.823252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758b847-044b-41b0-a784-d4b34838c550 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:45.875609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:45.876472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:45.876896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:45.877310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:45.878227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:45.878680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:45.880262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:45.908708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 347/710] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:33:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:45.914173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:45.918728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.034824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4cf566-8777-45d0-ab18-8ada2895055b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:46.043880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:46.081363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c03d97-075b-49bd-98a8-68f440b0a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:46.081977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c03d97-075b-49bd-98a8-68f440b0a397 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.082227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c03d97-075b-49bd-98a8-68f440b0a397 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.082497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c03d97-075b-49bd-98a8-68f440b0a397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.109530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.191656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c03d97-075b-49bd-98a8-68f440b0a397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:46.193111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c03d97-075b-49bd-98a8-68f440b0a397 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:46.198496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.207066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c622c898-48e5-4345-99f3-e05ed5b1dcd6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:46.281219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:46.281892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.282165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.282430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] PATH_INFO: `/v3/services/network` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.283014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:46.284386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:46.308666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:46.310746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:46.311554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:46.316529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 348/711] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:46 2022] GET /identity/v3/services/network => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:46.318114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:46.385908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 364/712] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:46.387222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee691264-2a9f-4ffb-9066-335bf7963c95 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.387512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee691264-2a9f-4ffb-9066-335bf7963c95 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.387789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee691264-2a9f-4ffb-9066-335bf7963c95 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.392285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 365/713] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:46.393106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee691264-2a9f-4ffb-9066-335bf7963c95 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:46.456855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f78c8-fee7-417b-ba3e-e354e7782688 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:46.467573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.503998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:46.505320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.505738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.506527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.507234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] RBAC: Adding query filter params (name=network) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:46.507624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:46.510857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974e1b9d-880e-46d2-a8f1-0af9cbb42ff0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:46.518074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 349/714] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:33:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:46.519629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.520042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.520294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.521833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee691264-2a9f-4ffb-9066-335bf7963c95 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:46.525551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.544484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:46.546012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:46.563515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:46.564150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.564430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.564703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.577359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:46.578979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:46.596001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:46.597081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:46.597989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:46.597989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:46.597989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:46.601827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 366/715] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:46.602802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:46.637801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 350/716] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:46.639333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:46.747742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ef123-8ff6-4347-aebc-40d63806138e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:46.749719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.774378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e14e5b6-3dd3-4ed4-a3ee-70c9840310ad None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:46.815116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:46.816078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.816566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.817079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.818080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] RBAC: Adding query filter params (type=network) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:46.818582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:46.820229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:46.824965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:46.825677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.825872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.826275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.826514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 367/717] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:33:46 2022] GET /identity/v3/services?type=network => generated 124 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:46.826767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:46.827063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:46.828412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec5b529-0d9a-4009-b54b-cb3ffadb4b12 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:46.857286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 351/718] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:46.862517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:46.866608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fae99-5d15-45d8-b805-272ce7cd7f50 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.866893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fae99-5d15-45d8-b805-272ce7cd7f50 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.867171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fae99-5d15-45d8-b805-272ce7cd7f50 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.870462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 352/719] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:33:46.884542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:46.886759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:46.888085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:46.909902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:46.921052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:46.922404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:47.043914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 353/720] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 05:33:47.045195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.049706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26717d3-1408-4240-844d-9804cca27a4b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.088992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.090006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.090433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.090851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.098535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:47.100317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:47.114059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:47.115236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:47.116244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:47.119026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 368/721] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:47.124803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.151996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:47.169525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01b1b6-5e04-4e35-9ad7-a0a35ba9cb5b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.240992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.242822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.243541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.244769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.264183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:47.275790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:47.276946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:47.278034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 354/722] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:47.285836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.298021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:47.344717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb694f-4a3f-44b6-a39b-cf2bfb58df20 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.396655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.397655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.397655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.397951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.398628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:47.398929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:47.401089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:47.417712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 369/723] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:47.417712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.431744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb868f-e625-4f88-b733-bd8238f4c752 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.467855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.468612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.468993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.469366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.470198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:47.470579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:47.472048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:47.486164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 355/724] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:47.487134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.503968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:47.532136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ddb90-6e2e-4403-90e5-ad4e7972779c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.569816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.570795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.571347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.572278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.583753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:47.590276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:47.591202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:47.592017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:47.594575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 370/725] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:47.595548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.596545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c8477-f5a3-4c93-b5e4-bbba72625b1b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.613951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:47.632768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.633494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.634457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.634802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.636169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=4e926d4a438e42ecaeab44f711e0b3c9) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:47.636169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:47.637644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:47.650360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 356/726] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/role_assignments?user.id=4e926d4a438e42ecaeab44f711e0b3c9&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:47.651506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.709739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870dc09-bbee-43a8-a427-ea0a9a859523 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.760667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.763081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.763846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.765035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.787657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:47.789822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:47.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:47.818623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:47.819493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:47.819493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:47.819493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:47.819493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 371/727] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:47.821408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:47.827141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:47.889086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:47.890588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b4c94d-b876-4e80-88f4-c87bf0660d0c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:47.936243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:47.937229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:47.937760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:47.938258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:47.939162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:47.939672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:47.941303 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:47.951369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 357/728] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:47.958793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.072685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640ba6a8-23a4-4633-908b-9057dcef696c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.110987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.111594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.111865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.112132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.112769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:48.113166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.114640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321190c0-decc-4799-8e78-f62ac73fce5b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.120638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2e3d5-8a26-407e-b2c8-15f41d160236 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.127585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 372/729] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:48.128873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2df8-8376-412b-b2d5-3e43543463fd None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.129182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2df8-8376-412b-b2d5-3e43543463fd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.129519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5a2df8-8376-412b-b2d5-3e43543463fd None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.135409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 373/730] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:48.136709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.136989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.137280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.156816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:48.158011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:48.166159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.167368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.167647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.167917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.179873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:48.183622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.185104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:48.201741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:48.202668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:48.203535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:48.203535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:48.203535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:48.203535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 358/731] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:48.204156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.261817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 374/732] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:48.262661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.350034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:48.371439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc68a2f-2aa9-4c0c-99c7-f6bded4e17f1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.408266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a96c87-3eb9-4611-9b0e-d57484580723 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.453043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.454980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.456103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.456606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.458228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:48.459677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.461260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.466223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.467180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.467662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.468132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] PATH_INFO: `/v3/users/swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.477708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.480234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.491748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 375/733] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:48.492692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:48.493659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:48.494572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:48.497244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 359/734] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:48.501720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.503238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.566256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:48.636792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84db3f96-1ace-4843-a748-39ee34ca1ffb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.679054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d638c76-c08a-4030-8540-770578b69ce3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.680505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:48.685497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.686801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.687418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.687905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.689283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:48.689851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.691888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.720568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 360/735] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 05:33:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:48.720568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.722545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.723325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.723761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.724136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.733253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.735089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:48.748477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:48.749671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:33:48.750791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:48.752666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 376/736] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:33:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:48.753736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.850930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e606ccc7-3928-488b-80f6-552aa9cd3a53 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.865501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:48.871616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b52f2a-1553-470b-b08f-cd663c559c5f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:48.895184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.895723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.896005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.896278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.896922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.898317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:48.909296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fff2e-97b9-498d-bbc2-6b1b4be58b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:48.910145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fff2e-97b9-498d-bbc2-6b1b4be58b3c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:48.910264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fff2e-97b9-498d-bbc2-6b1b4be58b3c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:48.910560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fff2e-97b9-498d-bbc2-6b1b4be58b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:48.915430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 361/737] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:33:48 2022] POST /identity/v3/services => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:48.923260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:48.948446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:48.995687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fff2e-97b9-498d-bbc2-6b1b4be58b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:48.997320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fff2e-97b9-498d-bbc2-6b1b4be58b3c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.081952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7db30-0213-4550-8242-cc5cf2a8298d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.125409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.126063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.126345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.126621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.127354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:49.127664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.128525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 377/738] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:49.129128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b051a2c-7c68-44cb-b38c-2009f0262a09 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.130185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931bcdd-1eaf-4726-9235-a3259b652e4f None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.130655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931bcdd-1eaf-4726-9235-a3259b652e4f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.131306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b931bcdd-1eaf-4726-9235-a3259b652e4f None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.133816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 378/739] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:49.134671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931bcdd-1eaf-4726-9235-a3259b652e4f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.155187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 362/740] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 05:33:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:49.156514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.156904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.157282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.176891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:49.178278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:49.197491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:49.221146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:49.272120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 363/741] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:49 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 05:33:49.273004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.275058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b931bcdd-1eaf-4726-9235-a3259b652e4f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.315658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.316835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.317317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.317904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.331277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.333660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:49.352155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:49.353929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:49.354796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:49.354796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:49.354796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:49.365742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 379/742] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:49.366567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.486305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a763ab17-bd96-413c-813e-78056c2323c7 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.525660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1c560-56a0-47a8-ab54-50776e9e658b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.540746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:49.547165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.548268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.548807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.549086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.563215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.564173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.564665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.565136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.566014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:49.566517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.568081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.581091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 380/743] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:49.589341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.615274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:49.699050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.700626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.716911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b156d-6862-4775-8666-3979f40ac233 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.753803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.754854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.755306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.755755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.763744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.765506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:49.778194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:49.779091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:49.779953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:49.782457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 381/744] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:49.783413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.808626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 364/745] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:49 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 05:33:49.809529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.820222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:49.868256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:49.894492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c265a1d-a607-4fed-aa97-9940d6b41c24 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.915363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed278a0-eebe-4a57-8cfa-be4817718492 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:49.930603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055ca50-c643-47e9-aa93-2d56cbffd36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.931287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055ca50-c643-47e9-aa93-2d56cbffd36e None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.931491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055ca50-c643-47e9-aa93-2d56cbffd36e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.931762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055ca50-c643-47e9-aa93-2d56cbffd36e None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.932501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055ca50-c643-47e9-aa93-2d56cbffd36e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.933776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e055ca50-c643-47e9-aa93-2d56cbffd36e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.949994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 382/746] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:33:49 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:49.951195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f247e-f505-4f1c-9337-13df5315ee86 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.951474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f247e-f505-4f1c-9337-13df5315ee86 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.951742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f247e-f505-4f1c-9337-13df5315ee86 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.953064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 383/747] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:49.953648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f247e-f505-4f1c-9337-13df5315ee86 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:49.964666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:49.965762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.967050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.967571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:49.969164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:49.969777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:49.972456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46e070b-398e-40a7-8ef5-d5876be5facb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:49.993751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 365/748] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:49.996034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:49.996593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:49.997097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.022687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:50.024563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:50.091388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f247e-f505-4f1c-9337-13df5315ee86 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.098163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:50.123494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 366/749] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:50.124404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.138012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.138670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.138952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.139222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.152370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:50.154158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:50.176698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:50.177677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:50.178558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:50.178558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:50.178558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:50.179143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 384/750] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:50.187001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.195477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:50.268646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde5e30-1b4f-4979-9243-e0a45476fc7a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.321122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c032f6-8cad-4776-a0f7-31996320a1d9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.368083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.369284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.369599 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.369999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.370483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.370755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cdca2e846d9340a5ab6ee8356115a6c7) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:50.371073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:50.371279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.371607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.371942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.372546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:50.372708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:50.373054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:50.374526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:50.389773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 385/751] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:50.395307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 367/752] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/role_assignments?user.id=cdca2e846d9340a5ab6ee8356115a6c7&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:50.397960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.410028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:50.427088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.558791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0ff3c-c56d-4e10-a03f-614a2dcff01c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.562022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:50.616815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa93ea-d5ae-485f-9045-20b212cdf632 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.619554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.620208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.620486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.620760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.646344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:50.651810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:50.652727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:50.653539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:50.655524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 368/753] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:50.661276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.688083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.688737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.688993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.689260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.704603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:50.710009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:50.730079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:50.731007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:50.731858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 386/754] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:50.742597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.803591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ebd6d0-81ab-415f-bff2-1216081c1f12 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.851661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.852275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.852580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.852852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.853512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:50.854094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:50.855845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:50.878024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:50.878214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 369/755] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:50.878760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.892333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69227496-d04a-421d-8f94-59de09cae703 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:50.929450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:50.930442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:50.931480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:50.932239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:50.933254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:50.933799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:50.935703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:50.950408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 387/756] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:50.959036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:50.968443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:51.093472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191872c-607b-4f23-949f-fe785850ead6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:51.117670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e65b3-ac5a-4ce2-822e-944217e69471 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:51.134531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:51.135261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.135546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.135799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.147492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:51.147648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:51.149948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:51.157516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86294033-a210-406c-925d-5d60ec422207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:51.158448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86294033-a210-406c-925d-5d60ec422207 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.158929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86294033-a210-406c-925d-5d60ec422207 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.159403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86294033-a210-406c-925d-5d60ec422207 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.160324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86294033-a210-406c-925d-5d60ec422207 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cca6991f4cff4e8b94c3d8b727fb6633) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:51.160924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86294033-a210-406c-925d-5d60ec422207 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:51.164358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86294033-a210-406c-925d-5d60ec422207 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-513779d8-08b0-4eb1-9455-f09cd9b68f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:51.165362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:51.166282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:51.167135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:51.167135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:51.167135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:51.171282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 370/757] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:51.172482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26da0-3f32-4c89-881d-bd040b4df79e None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.172759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26da0-3f32-4c89-881d-bd040b4df79e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.173024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a26da0-3f32-4c89-881d-bd040b4df79e None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.173957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 388/758] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:50 2022] GET /identity/v3/role_assignments?user.id=cca6991f4cff4e8b94c3d8b727fb6633&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:51.174523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 371/759] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:51.175180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86294033-a210-406c-925d-5d60ec422207 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:51.180260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.180567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.181330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.201632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:51.202833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:51.287311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86294033-a210-406c-925d-5d60ec422207 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:51.294945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 372/760] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:33:51.324790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:51.325848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.326336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.326850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.327695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:51.328195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:51.329821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:51.343628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 389/761] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:51.350970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:51.369231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:51.488426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32d6725-86e2-4fdc-857b-f226552ca3f4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:51.524972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59149c3-e031-4820-845d-553098398b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:51.525666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59149c3-e031-4820-845d-553098398b9f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.525950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59149c3-e031-4820-845d-553098398b9f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.526218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59149c3-e031-4820-845d-553098398b9f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.533635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59149c3-e031-4820-845d-553098398b9f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:51.535072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59149c3-e031-4820-845d-553098398b9f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-a59149c3-e031-4820-845d-553098398b9f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:51.546593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:51.547506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:51.548428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 373/762] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:33:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:51.553517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:51.619327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:51.676174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ed2e3e-bf59-4d57-a8c1-d82d24b64a59 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:51.711134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:51.711751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.712550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.712820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.713552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:51.713877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:51.715313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:51.734498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 390/763] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:33:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:51.739967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59149c3-e031-4820-845d-553098398b9f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:51.838938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:51.878163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59149c3-e031-4820-845d-553098398b9f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:51.879421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:51.934788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:51.935808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:51.936322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:51.936794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:51.948615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:51.950362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:51.965348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:51.966317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:51.967175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:51.967175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:51.967175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:51.969264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 374/764] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:51.975252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.096271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d3d4d-81c8-44a3-a733-2d8c83f1e0e9 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.116335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875d8dd-61bb-4f0c-964e-28134efba4f7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.120196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:52.131599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:52.138403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.139087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.139367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.139639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.140736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:52.141049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.142766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.159357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 375/765] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:52.165590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.175399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.176350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.176905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.177373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.178249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.179815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:52.193072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:52.194132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:52.194970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:52.201759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 391/766] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 05:33:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:52.203628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.276732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e15a2a-419e-4bf9-80ab-a2a35cc871f5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.320252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.320931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.321239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.321516 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.330542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.332267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e8aeba21-e6c0-4e06-8e6a-1f33780fef23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:52.347239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:52.348137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:52.349008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 376/767] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:52.349481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f770f1-7542-49d8-bb83-a2e8c740a392 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.354927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cb8f-d260-4ffc-b009-bd0f0b650f16 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.356196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cb8f-d260-4ffc-b009-bd0f0b650f16 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.356531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1cb8f-d260-4ffc-b009-bd0f0b650f16 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.362158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 377/768] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:52.363164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cb8f-d260-4ffc-b009-bd0f0b650f16 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.398509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.399140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.399531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.399807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.400660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:52.400972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.402548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa114ef-c85f-4617-be26-c0464c595942 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.408820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 392/769] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:52.410117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.410272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:52.410410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.410657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.432126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:52.434242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:52.468107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:52.486141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1cb8f-d260-4ffc-b009-bd0f0b650f16 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.507806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 393/770] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:52.508724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.539107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.539776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.540759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.541089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.542324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:52.542636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.544117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.571541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 378/771] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:52.578095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.647203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f8a329-ba3a-4b72-abb0-fd9fcd593bc9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.707819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.708843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.709335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.710268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.711101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:52.711599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.714660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973643ca-4c1d-407a-b9fb-db1b2359d617 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.721948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 394/772] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:52.723323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f47cf-f26f-49ef-a10a-247ff7b62a96 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.724326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f47cf-f26f-49ef-a10a-247ff7b62a96 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.724800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1f47cf-f26f-49ef-a10a-247ff7b62a96 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.731864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 395/773] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:52.731864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f47cf-f26f-49ef-a10a-247ff7b62a96 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.739352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:52.742362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a783449c-6a0b-492d-9ad6-95f400c89f7b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.767856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:52.782933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594acfde-d300-48d4-bde7-5cf5da9c0171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.784043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594acfde-d300-48d4-bde7-5cf5da9c0171 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.784527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594acfde-d300-48d4-bde7-5cf5da9c0171 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.784987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594acfde-d300-48d4-bde7-5cf5da9c0171 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/4e926d4a438e42ecaeab44f711e0b3c9/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.787150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594acfde-d300-48d4-bde7-5cf5da9c0171 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=4e926d4a438e42ecaeab44f711e0b3c9, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.788901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594acfde-d300-48d4-bde7-5cf5da9c0171 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.806091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 379/774] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:52 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/4e926d4a438e42ecaeab44f711e0b3c9/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:33:52.807839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.808313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.808872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.843195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:52.844473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:52.878517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1f47cf-f26f-49ef-a10a-247ff7b62a96 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:52.933820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:52.933820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:52.933820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:52.933820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:52.934266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] RBAC: Adding query filter params (interface=public, service_id=3752a079eaa44ea791404b677fca4447, region_id=RegionOne) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:52.934266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:52.934266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:52.934739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 380/775] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:52.936064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:52.944759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 396/776] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/endpoints?service_id=3752a079eaa44ea791404b677fca4447&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:52.950832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.066000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.071911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.072992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d72a0-c2e5-4154-8399-96e01b1fa131 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.082371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3722bb-7cbc-49c8-923c-cedacf35ed9c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.115564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.116206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.116498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.117006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.117643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.120030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.123102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.123928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.124364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.124782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] PATH_INFO: `/v3/services/network` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.125503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.126194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 397/777] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:53.127245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:53.139735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:53.140997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:53.142067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:53.144579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 381/778] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:52 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:53.151382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.254259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.275285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eeab5d-9ead-4460-b22e-83ae296dc276 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.317166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.317743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.318024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.318299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.318954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] RBAC: Adding query filter params (name=network) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:53.319258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.320890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.327376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 398/779] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:53.334740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.363136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.434760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701ab45-54b3-469d-a7c8-79054bb17df1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.467916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d44c7-0a6e-4d37-bb00-dbad3f11f83a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.479742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.480851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.481132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.481405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.495847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:53.502998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:53.503895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:53.504720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:53.505332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 382/780] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:53.513064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.513784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.514126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.514461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.514766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.515312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] RBAC: Adding query filter params (type=network) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:53.515679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.517115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.532169 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 399/781] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:53.534408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.546947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.666019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.668696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e813289-7d76-4b40-9b28-699e201dc300 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.673828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b761626-99c7-436d-8a72-264d9fcfd1e4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.706639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.707635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.708235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.708715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.710134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:53.710614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.713437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb1109b-6ab3-474e-a58b-2f29a8b0e53f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.717952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.718635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.718983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.719341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.720718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] RBAC: Adding query filter params (interface=public, service_id=8df0278210be47d1b688dea713cd6c40, region_id=RegionOne) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:53.721099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.722855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.724447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 383/782] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:53.726532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bcbf9-917f-4c09-940b-31563ca75d53 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.727068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bcbf9-917f-4c09-940b-31563ca75d53 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.728149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bcbf9-917f-4c09-940b-31563ca75d53 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.733611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 400/783] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/endpoints?service_id=8df0278210be47d1b688dea713cd6c40&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:53.733611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.737881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 384/784] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:53 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:53.739041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bcbf9-917f-4c09-940b-31563ca75d53 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.835801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.858489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06540238-d6ff-49c1-8547-b20291a8e8e2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.865466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bcbf9-917f-4c09-940b-31563ca75d53 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:53.901167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.901829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.902109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.902382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.911634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be20a1a-b335-4bd5-88d9-1cdfd9228393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:53.912831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20a1a-b335-4bd5-88d9-1cdfd9228393 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.912959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.913443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20a1a-b335-4bd5-88d9-1cdfd9228393 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.913932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be20a1a-b335-4bd5-88d9-1cdfd9228393 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.914359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.915695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be20a1a-b335-4bd5-88d9-1cdfd9228393 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:53.917861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be20a1a-b335-4bd5-88d9-1cdfd9228393 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:53.924635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 385/785] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:53.926444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:53.926941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:53.927433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:53.931311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:53.932262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:53.933129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:53.933129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:53.933129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:53.933129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 401/786] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:53.938855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:53.941707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:53.951661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:53.953454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:54.070537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb2cf15-d692-4037-a55a-e3d08775e171 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:54.089300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 386/787] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:54.110598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:54.111376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:54.111772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:54.112140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:54.112887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:54.113289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:54.114800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:54.128673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 402/788] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:54.134146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:54.226825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:54.260417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa41358-ec8f-4714-b9f9-18f3af082eb3 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:54.304984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:54.305666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:54.305949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:54.306220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:54.316731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:54.318215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:54.333069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:54.334268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:54.335431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 387/789] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:54.342489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:54.474106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:54.499531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:54.522780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d11841-2c98-4cc1-b9d8-b02b53251e1c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:54.585265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:54.586220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:54.586657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:54.587729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:54.588717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:54.589195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:54.590806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:54.623642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7444c461-1343-403d-9d7c-4a2b52d3d669 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:54.626010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 403/790] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 05:33:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:54.631891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:54.663985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:54.671993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:54.672649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:54.673018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:54.673308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:54.686285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:54.691505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:54.692423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:33:54.693237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:54.697185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 388/791] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:33:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:54.699767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:54.813085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69b5bf-4d15-48ca-8da0-3503db018cc2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:54.851190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:54.851942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:54.852231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:54.853043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:54.854133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:54.854439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:54.856017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:54.869723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 389/792] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:33:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:54.871755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:54.933960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:54.985313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32383b86-18c9-44c0-81f5-060ec148668a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.003532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16b2480-a6cf-4a57-a4f9-f3ad76203c8a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.028407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.029076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.029358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.032431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.048710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:55.058070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.059549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.083203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.083844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.084126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.084403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:55.087895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:55.088827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:55.089744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:55.089744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:55.089744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:55.103830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 390/793] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:55.106063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.109977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.111455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-64335eee-3334-4008-8dd5-df43f3b176dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:55.140564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:55.142025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:55.143171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:55.143171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:55.143171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:55.143171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 404/794] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 510 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:55.146907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f09faf-a865-425c-8f57-30893c4c3c8d None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.147181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f09faf-a865-425c-8f57-30893c4c3c8d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.147658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f09faf-a865-425c-8f57-30893c4c3c8d None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.150196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 405/795] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:55.152047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f09faf-a865-425c-8f57-30893c4c3c8d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.248371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ca1e1-9e9d-4348-8aac-601344193d8f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.251798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:55.287281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f09faf-a865-425c-8f57-30893c4c3c8d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.299011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.300352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.301001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.301617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.302481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:55.303129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.305540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a77377-b495-49c1-8da8-ae6159892e23 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.322465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 391/796] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:55.324030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.324471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.324897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.335727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.336336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.336615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.336918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.337620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:55.337926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.339459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.345100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:55.346522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:55.357577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 406/797] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:55.357577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.390002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:55.445587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 392/798] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:55.445587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.486113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d12df-3d62-4a21-9427-d5514990cf93 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.531650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.532283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.532558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.532827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.543474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.545205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.556045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-fd0a308d-d2e3-4f6b-86fc-b869a092784f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:55.562129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:55.563120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:55.564047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 407/799] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:55.565144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de359d0-66d6-47a4-98c1-62fe81f3574c None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.565144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de359d0-66d6-47a4-98c1-62fe81f3574c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.565413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de359d0-66d6-47a4-98c1-62fe81f3574c None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.566891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 408/800] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:55.567753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b23007-6a64-41a4-be75-455db7e67dd4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.569040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de359d0-66d6-47a4-98c1-62fe81f3574c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.607312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.608370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.608955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.609497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.617419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.619192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-d23122b0-e78d-484f-8ac9-286eefd93b94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:55.632043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:55.633015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:55.633950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:55.641063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 393/801] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:55.642909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.643394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.643866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.663376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:55.665710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:55.680063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:55.692126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de359d0-66d6-47a4-98c1-62fe81f3574c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.730820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.731428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.731704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.731972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.732701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:55.733007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.734462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.740715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 394/802] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:55.741406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.761156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 409/803] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:55.762414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.835911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:55.859754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47ffba-1176-4b43-94a4-8e6db31e1e18 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.890896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e89eac-0efe-4aa2-b819-6a68b862a7cb None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:55.911430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.913196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.913747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.914464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.915676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:55.916182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:55.918451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:55.927503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:55.928235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:55.928535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:55.928888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:55.939435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 395/804] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:55.939435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.945781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:55.952201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:55.953157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:55.954013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:55.957479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 410/805] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:55.958521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:55.975251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.058290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460f6e4-1ca0-4673-9265-38a75f709d7e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.075194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1779b7db-4f03-4638-8724-59fa3bcdfa74 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.098595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.099226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.099500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.099770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.111436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.112826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.115386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.122794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.123468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.123748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.124016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/cdca2e846d9340a5ab6ee8356115a6c7/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.126002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=cdca2e846d9340a5ab6ee8356115a6c7, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.127441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:56.128716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:56.129650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:56.130519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:56.130519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:56.130519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:56.130519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 396/806] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:56.131141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.148050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 411/807] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:55 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/cdca2e846d9340a5ab6ee8356115a6c7/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:56.154733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.239553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.266581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538df485-3b7e-4d06-abe7-2e19881049e6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.279858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b96c8f-d326-45eb-89f9-5b4c1042b309 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.306533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.307672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.307955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.308227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.309178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:56.309502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.311656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.321380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.322046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.322539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.322644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 397/808] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:56.322742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.323704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.323823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.324576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ff686642-59b5-43bf-a1ee-8684e2646fb8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:56.334038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:56.335019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:56.337329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 412/809] 10.222.0.73 () {56 vars in 1129 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:56.338380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a97ba1-f7e8-4f21-8718-ee31c953c658 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.338380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a97ba1-f7e8-4f21-8718-ee31c953c658 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.338380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a97ba1-f7e8-4f21-8718-ee31c953c658 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.338916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 413/810] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:56.340396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a97ba1-f7e8-4f21-8718-ee31c953c658 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.393742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.456165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bad7d9-ef71-491d-a007-8fdc07c9ac30 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.478387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a97ba1-f7e8-4f21-8718-ee31c953c658 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.509947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.510877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.511295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.511797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.512902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:56.513391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.515734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.533276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.534275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.534745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.535167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.535614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 398/811] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:56.537063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.545633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.555822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.557403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-562ae284-dc63-4a0e-b02d-535f4eaa42f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:56.574332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:56.575478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:56.576544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:56.576544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:56.576544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:56.576544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 414/812] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:56.576833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.582221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.582221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.596330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:56.597534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:56.657740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81f527-9940-4349-8442-27533d4a4d48 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.669626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 415/813] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:56.681733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.688503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.710208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.711583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.712674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.713025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.713991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:56.717925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.720433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.733044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 399/814] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:56.734134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.806714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48c7eb-a092-45ed-88f2-063ef7aafa86 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.831334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:56.854799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.855334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582b3de-d32c-4839-b69b-f8fa303507b5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:56.855824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.856252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.856671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.864357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.866144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:56.879096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:56.880189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:56.881245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:56.883381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 416/815] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:56.884326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.901052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:56.901977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:56.902392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:56.902760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:56.903932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:56.904613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:56.906440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:56.922529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 400/816] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:56.923444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:56.974966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:57.004437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23cb1c6-a2cc-45be-9945-3afb65aba656 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.042394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.043110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.043456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.043794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.044507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:57.044878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.046374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.057123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 417/817] 10.222.0.73 () {56 vars in 1124 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:57.057123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.105450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:57.157028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d545d1-10a7-4980-a9c5-eb96eb641053 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.171664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1001fd08-47d6-44d6-a78a-de83c5a25d48 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.192078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.192691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.193056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.193398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.194185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:57.194552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.196058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.208221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 401/818] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:57.208917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.215939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.216681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.217029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.217384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.226792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.229323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:57.242079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:57.243680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 418/819] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:33:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:57.244653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.335977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80caeb6-9887-40d1-9e43-28addb670c50 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.337806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:57.351100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:57.355314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c273ce11-ae4c-4d64-bc25-822c9a38611f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.372864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.373663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.373962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.374302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] PATH_INFO: `/v3/services/network` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.374956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.376395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:57.385205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:57.386396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 05:33:57.387427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:57.388261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 402/820] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:57 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:57.388964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.400419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.401034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.401314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.401611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.402274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.403636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.433594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 419/821] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:33:57 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:57.433594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.559111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b797073-2b6c-4f59-add4-00f71eaf0185 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.602307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc235c7e-82ea-4f03-a8b2-5bb16a64cb98 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.606783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.607679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.607961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.608234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/cca6991f4cff4e8b94c3d8b727fb6633/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.628095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=cca6991f4cff4e8b94c3d8b727fb6633, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.630345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.654573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 403/822] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:33:57 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/cca6991f4cff4e8b94c3d8b727fb6633/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:33:57.654573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.655692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.656333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.656614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.656885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.657655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:57.657964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.659449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.696376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:57.701662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 420/823] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:33:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:57.710354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.732254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:57.872389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ea90d3-ebf7-4b6f-be46-3cca56b13dda None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.884922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce047a37-cf8a-43a3-9410-9ba5f7b7bbb9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:57.919389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.920277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.920687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.921331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:57.921867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] RBAC: Adding query filter params (name=network) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:57.922339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:57.923849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:57.928790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 404/824] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:33:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:57.938576 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:57.939803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf37f4f-cf23-406c-9518-0e8ea817c7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:57.940587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf37f4f-cf23-406c-9518-0e8ea817c7f9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:57.941010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf37f4f-cf23-406c-9518-0e8ea817c7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:57.941819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf37f4f-cf23-406c-9518-0e8ea817c7f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.022387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf37f4f-cf23-406c-9518-0e8ea817c7f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.023929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf37f4f-cf23-406c-9518-0e8ea817c7f9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.032087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:58.037980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:58.053463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e89bcc1-3496-475e-a4a6-4b0b8cb14218 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.091159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.091779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.092060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.092331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.092964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] RBAC: Adding query filter params (type=network) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:58.093267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.094675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2817761e-5fd8-4e5d-b38e-a38f89b81d63 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.100971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 405/825] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:33:57 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:58.102188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf895ca-c263-4b3d-954c-9ab9aee92ddf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.103033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf895ca-c263-4b3d-954c-9ab9aee92ddf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.103308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf895ca-c263-4b3d-954c-9ab9aee92ddf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.109479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 406/826] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:58.110341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf895ca-c263-4b3d-954c-9ab9aee92ddf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.140100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 421/827] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:57 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:33:58.141336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.141713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.141916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.160956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:58.162207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:58.218215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf895ca-c263-4b3d-954c-9ab9aee92ddf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.234635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 422/828] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:58.235577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.261837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.262461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.262742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.263014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.263678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.265286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.279606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:58.309955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:58.339477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 407/829] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:33:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:33:58.351275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9c8569-45c5-4d9c-bff0-67dc71f46a21 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.398609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.399240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.399578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.399855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.411819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.413350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:58.429845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:58.431739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:58.433051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:58.433051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:58.433051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:58.433051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 423/830] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:58.438450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.531431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.618605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a8e817-0aea-4b62-8f45-175743357f73 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.663531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:58.673535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.674174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.674454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.674725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.675372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:58.675675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.677053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.687529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f9ef2-dbf7-4548-bfa6-de6d492bafa9 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.691461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 408/831] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:58.697417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.731870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.733317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.733774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.734186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.747203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:33:58.754206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:58.755382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:33:58.756387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:58.759661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 424/832] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:33:58.765589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.774788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:58.836557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32a8e48-44d2-4c3f-8cb5-4d2a6ec6aadd None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.883410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.884816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.885620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.886125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.895563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.897921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:33:58.913027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:58.914198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:33:58.915334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:58.917731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba930a4-c8b6-4458-ab39-a88722b61aa8 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:58.920768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 409/833] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:58.922072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:58.967062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:58.967675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:58.967951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:58.968216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:58.968882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:58.969183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:58.970578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:58.979839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 425/834] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:58.986197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:59.003433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:59.045374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627e313-e950-49ee-aa71-3661f2076aeb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.069148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:59.095069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.096159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.096583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.097153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.098380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:59.098876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:59.100463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:59.117039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 410/835] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:33:59.123999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:59.124219 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4d9ed-3f23-41b6-8620-e751e93d4fab None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.164484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.165630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.165972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.166324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.178381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:59.179890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:33:59.196568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:59.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:33:59.198440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:33:59.198440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:33:59.198440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:33:59.202622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 426/836] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:33:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:59.203978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:59.233900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7098ab-cd71-4b5b-ab60-31082e104947 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.269814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.270799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.271300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.271760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.272669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=4e926d4a438e42ecaeab44f711e0b3c9) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:59.273166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:59.275380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:59.275514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:59.290503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 411/837] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:33:59 2022] GET /identity/v3/role_assignments?user.id=4e926d4a438e42ecaeab44f711e0b3c9&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:59.326143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2145689f-787d-4b5f-bea6-76b2d15138d8 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.367982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.368607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.368890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.369193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.370194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:59.370506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:59.371909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:59.390761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 427/838] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:59.394824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:59.447471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:59.507882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b4d88-361d-470f-8a99-057f876e36ef None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.548412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.549052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.549333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.549644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.558073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:59.559526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2dacf5ec-f685-453f-aa2a-66f3d624c815 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:33:59.571648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:33:59.572684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 05:33:59.573662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:33:59.577675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 412/839] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:33:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:33:59.579237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:59.662988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e828-a803-46bd-90b9-a1d4d3531caf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.663338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e828-a803-46bd-90b9-a1d4d3531caf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.663535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4636e828-a803-46bd-90b9-a1d4d3531caf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.665538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 413/840] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:33:59.670988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e6cce-a6a2-4e80-bbed-af933933933a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.671274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e6cce-a6a2-4e80-bbed-af933933933a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.671550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07e6cce-a6a2-4e80-bbed-af933933933a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.691316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f07e6cce-a6a2-4e80-bbed-af933933933a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:59.691616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de3d3f-5ed9-45ae-aec7-107b8cbd4bdc None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.692524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f07e6cce-a6a2-4e80-bbed-af933933933a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:59.720187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:59.735360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.736003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.736285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.736665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.737424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:33:59.737745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:33:59.739297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:33:59.759913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 428/841] 10.222.0.73 () {56 vars in 1135 bytes} [Wed Aug 24 05:33:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:33:59.765663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:33:59.790256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 414/842] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:33:59.884588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca7572-d123-4d70-8746-88cc7131f84a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:33:59.914069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893a02c-5bca-42e6-972b-61a0af771003 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.915020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893a02c-5bca-42e6-972b-61a0af771003 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.915802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3893a02c-5bca-42e6-972b-61a0af771003 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.917775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 415/843] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:33:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:33:59.926487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:33:59.927103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.927383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.927610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:33:59.927714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.928325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:33:59.928895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:33:59.952671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:33:59.954611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:33:59.980673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:33:59.991684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.006301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.007752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.046977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 416/844] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:33:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:00.061296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.121655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 429/845] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:33:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:00.127382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.183680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6bc0a-8719-40a3-8747-6e6a9683449c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.217598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.221422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.222727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.223010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.223400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.223988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.225597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:00.235318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:00.236288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:00.237116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 417/846] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:00.242280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.246091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d36558-1514-41c3-ad95-d7f00b82de23 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.281637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.290478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.291279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.291734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.292158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.307037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.308706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:00.328002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:00.328982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:00.329883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:00.329883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:00.329883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:00.331630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 430/847] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:00.332635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.375243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe36297-2ff0-4b2b-b1b4-deb3fe8b216d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.419428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.420049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.420531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.420807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.421440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:00.421781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.423151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.427602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 418/848] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:00.428381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.445695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1d3b8-3bb3-4056-b36a-5f0e05800b9a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.479570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.486049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.486867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.487307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.487574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.499057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:00.504299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:00.505224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:00.506081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 431/849] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:00.506744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.536302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.546327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080a275-c24e-467b-a5b7-efd8ec9064f2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.580057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.580674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.580949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.581221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.581891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:00.582193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.583559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.595810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 419/850] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:00.596613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.623016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348f87c-ffd1-4e1d-850a-12b3ec2f59ef None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.660465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.661074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.661356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.661657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.662348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:00.662589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.664047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.677597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 432/851] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:00.677597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.720124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b818e12-374b-4d51-8fcf-db0e4a801d9f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.759838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.761845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.762551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.762903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.763325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.764084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:00.764488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.766013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.775696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 420/852] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:00.781281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.801230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:00.827802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c734-e652-4bad-a23d-1bf7ed2f416b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.877354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.878027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.878307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.878579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.886424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.887869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:00.902790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:00.903705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:00.904573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 433/853] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:00.910387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:00.917692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e62eb2-3d30-4780-8aa4-8825139295b4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:00.960296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:00.961340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:00.962395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:00.962874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:00.974976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:00.976788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:00.993612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:00.994649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:00.995530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:00.995530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:00.995530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:01.002124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 421/854] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:01.003325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.047912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b65c8e-eb00-4c3a-94f6-d2156a20fb74 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.068734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:01.077716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:01.096974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.097869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.098154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.098426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.099574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:01.099910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.101673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.121830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18dfa16-22d7-4129-81ca-ec73fad4b599 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.125185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 434/855] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:01.127252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.163342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.164400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.165579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.166216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.167079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:01.167585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.169212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.187625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 422/856] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:01.188655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.273803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041863b4-4483-4cc6-bbf9-8ebbf1c4094e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.316039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465cc259-91ad-4803-a57a-b2bc946a6f03 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.321615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.322288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.322576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.322850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.323579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cdca2e846d9340a5ab6ee8356115a6c7) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:01.323886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.325321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802cafcb-2904-4481-8348-6453eafa3ac3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.337743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 435/857] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/role_assignments?user.id=cdca2e846d9340a5ab6ee8356115a6c7&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:01.360030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:01.366462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.367019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.367658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.368472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.375373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04b7e1-5266-49b5-b315-b37eacffc52f None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.375690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04b7e1-5266-49b5-b315-b37eacffc52f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.375931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04b7e1-5266-49b5-b315-b37eacffc52f None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.377506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 436/858] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:01.381836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.384357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.384638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.384913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.386104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:01.400414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:01.401676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:01.402603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:01.404812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:01.405739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 423/859] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:01.406101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:01.414187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.455734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:01.506579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 437/860] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:01.542281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a2c64-49ad-4844-823f-45876b92c321 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.581784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.582397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.582679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.582953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.583683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:01.583988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.585432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.603140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 424/861] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:01.609095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.716777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a071e3-eab5-4af9-959e-ee322bc37a88 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.754982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.755972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.756476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.756932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.783888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.805906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:01.839030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:01.848396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.851258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.898655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c95108-86f5-4eb5-8b38-8580b9ef98e5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:01.935926 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:01.936644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:01.936927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:01.937202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:01.937877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:01.939319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:01.948947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:01.949899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:01.950765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:01.955009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 425/862] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:01.956017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:01.998729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 438/863] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:34:02.000044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:02.086449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:02.097683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3012954-08ec-4264-80ee-f385823df56c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.108881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:02.132471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5445cc72-a9f9-464f-90fe-958529cd73a1 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.138933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.139664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.140097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.140471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.141204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:02.141638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.143223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.153206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 426/864] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:02.154417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:02.177391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.178808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.179237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.179651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.199446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.200900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:02.219408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:02.220379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:02.221233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:02.221233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:02.221233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:02.221233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 439/865] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:02.226989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:02.287516 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0e4fe-3266-4f42-8da0-84ed7f0f0f19 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.325057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.325838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.326229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.326571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.327883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:02.327883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.329268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.341718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 427/866] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:02 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:02.363302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6816fb-b3e1-4b35-8dbb-cdaf6c69635e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.391290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:02.405333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.406395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.406852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.407318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.408155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:02.408639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.410234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.425332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 440/867] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:02.431382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:02.474027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:02.545717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f574337-2694-41c6-98ad-22b25b8a084e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.583887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.584533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.584812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.585081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.593196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.594674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0cd53fb0-fdde-4076-99b6-3bfccd0f4aa8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:02.606520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:02.607405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:02.608283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 428/868] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:02.615093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:02.647671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb5ce2-a4f2-480b-9478-d08c085ea4b6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.648012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb5ce2-a4f2-480b-9478-d08c085ea4b6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.648295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb5ce2-a4f2-480b-9478-d08c085ea4b6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.650223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 429/869] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:02.655473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.655756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.656030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.674438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:02.675629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:02.742138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be5b2e-dc98-4870-b47f-5fcb0cca9dc2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.757639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 430/870] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:02 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:02.787107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.788053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.788537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.789011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.789971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:02.790494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.792477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.804934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 441/871] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:02.810731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:02.839575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:02.920613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306f705-b805-4667-8ab3-f83f160f0d9b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:02.958320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:02.959288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:02.959769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:02.960261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:02.961221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cca6991f4cff4e8b94c3d8b727fb6633) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:02.962299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:02.964020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19357c1-1aa1-4136-9296-a81a489ba8b9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:02.970769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 431/872] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:02 2022] GET /identity/v3/role_assignments?user.id=cca6991f4cff4e8b94c3d8b727fb6633&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:03.027161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:03.080887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:03.143833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620111-01be-4abd-9bee-a4965360384e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:03.181364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13060eb1-cba2-4a7b-a280-dabb280d33d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:03.182232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13060eb1-cba2-4a7b-a280-dabb280d33d8 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:03.182719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13060eb1-cba2-4a7b-a280-dabb280d33d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:03.183135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13060eb1-cba2-4a7b-a280-dabb280d33d8 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:03.183923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13060eb1-cba2-4a7b-a280-dabb280d33d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:03.185486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13060eb1-cba2-4a7b-a280-dabb280d33d8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:03.205647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 442/873] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:03 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:03.445023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d11357-b2e2-407d-b53b-c8f34921080e None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:03.445513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d11357-b2e2-407d-b53b-c8f34921080e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:03.445513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d11357-b2e2-407d-b53b-c8f34921080e None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:03.447712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 432/874] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:03.453135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:03.453695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:03.453780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:03.480782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:03.482022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:03.563010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 443/875] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:03 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:03.838582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d11357-b2e2-407d-b53b-c8f34921080e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:03.937156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:03.949514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d11357-b2e2-407d-b53b-c8f34921080e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:03.986712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:03.987623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:03.988064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:03.988478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:03.989198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:03.990724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:03.999872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:04.001086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:04.002152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:04.004447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 433/876] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:04.010700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:04.038934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:04.118043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aec903-d411-4e20-bf89-c8f4b0d4c37e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:04.154811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:04.155492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:04.155777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:04.156051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:04.156845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:04.157155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:04.158544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47ff401-61e3-42e1-ab84-988dfc3edf04 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:04.168680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 444/877] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:04.170040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:04.281122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5877b35b-4877-4b35-9a98-de64143306c1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:04.318228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:04.318834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:04.319171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:04.319449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:04.320081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:04.320432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:04.321956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21bce9-5e0a-4bbe-92bf-9580559bf137 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:04.330557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 434/878] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:04 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:04.465305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c857-f53a-4109-b21c-df15e260526f None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:04.466326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c857-f53a-4109-b21c-df15e260526f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:04.466828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164c857-f53a-4109-b21c-df15e260526f None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:04.468615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 445/879] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:04.474330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4989-d240-43b8-b877-65d4355d1894 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:04.474753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4989-d240-43b8-b877-65d4355d1894 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:04.474953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c4989-d240-43b8-b877-65d4355d1894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:04.495627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7d8c4989-d240-43b8-b877-65d4355d1894 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:04.496825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7d8c4989-d240-43b8-b877-65d4355d1894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:04.512341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:04.570551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 435/880] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:04 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:04.837849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c857-f53a-4109-b21c-df15e260526f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:04.932458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:04.942250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164c857-f53a-4109-b21c-df15e260526f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:04.977181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060755f3-2d02-45bb-bf0c-be8b7715856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:04.977919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060755f3-2d02-45bb-bf0c-be8b7715856e None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:04.978196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060755f3-2d02-45bb-bf0c-be8b7715856e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:04.978465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060755f3-2d02-45bb-bf0c-be8b7715856e None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:04.979370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060755f3-2d02-45bb-bf0c-be8b7715856e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:04.980790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060755f3-2d02-45bb-bf0c-be8b7715856e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:04.999180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 446/881] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:04 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:05.060688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bf46e-d3a0-4bb0-a3e8-42b391e32476 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.061048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bf46e-d3a0-4bb0-a3e8-42b391e32476 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.061263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7bf46e-d3a0-4bb0-a3e8-42b391e32476 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.062972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 436/882] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:05.072201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff86d0-426b-4f26-b17d-79e3a6acfe5e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.072537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff86d0-426b-4f26-b17d-79e3a6acfe5e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.072913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff86d0-426b-4f26-b17d-79e3a6acfe5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.093334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-fbff86d0-426b-4f26-b17d-79e3a6acfe5e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:05.094539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fbff86d0-426b-4f26-b17d-79e3a6acfe5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:05.174369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 447/883] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:05.238505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354bdbc-4810-4437-940e-5572bc066d83 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.238796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354bdbc-4810-4437-940e-5572bc066d83 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.239042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5354bdbc-4810-4437-940e-5572bc066d83 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.240608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 437/884] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:05.246304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.246599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.246876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.274793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:05.276259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:05.346688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 448/885] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:05 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:05.453547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354bdbc-4810-4437-940e-5572bc066d83 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:05.565015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5354bdbc-4810-4437-940e-5572bc066d83 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:05.599559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:05.600116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.600395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.600667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.611856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:05.616707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:05.618407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:05.619393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 438/886] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 05:34:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:05.626838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:05.627034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:05.638123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:05.638287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:05.735923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a22961-6c37-48a6-9519-2f18a1f56e0f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:05.740799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bc5a4-576f-4be2-b536-880ee8cc04e8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:05.780507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:05.780711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:05.781122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.781230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.781399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.781492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.781684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.781778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.782270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:05.782376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:05.782744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:05.783638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:05.784274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:05.793990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:05.794960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:05.795834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 439/888] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:05.798124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 449/888] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:34:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:05.798911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:05.806703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:05.887252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:05.892250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:05.926300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6868a42-9613-4d72-8534-e3d8dc87dc2d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:05.936570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ed5277-118d-4121-8222-51ae521f9405 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:05.970047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:05.971495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.972028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.972461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.975318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:05.976022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:05.976411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:05.976775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:05.977606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:05.978012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:05.980590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:05.986425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:05.988161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:05.991087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 450/889] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:05.992077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:06.007021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:06.008210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:06.009806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:06.009806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:06.009806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:06.012325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 440/890] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:06.017609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.111121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ae5321-28b9-48a6-b6c7-018588f03588 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.132616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f405eb8f-40cd-4a1a-9100-5e5ee63455db None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.145644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.146205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.146487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.146757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.147409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:06.147711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.149678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.151583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:06.160969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 451/891] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:06.162110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.176275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:06.191408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.192370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.192901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.193365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.194403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:06.194901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.196582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.210838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 441/892] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:06.219548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.294940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986002d-3bdf-45f1-935d-2d4fc8ea36b1 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.330199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.330813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.331135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.331416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.332109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] RBAC: Adding query filter params (interface=public, service_id=9e2cb3b260f34f2ca66b65154fe9572e, region_id=RegionOne) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:06.332416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.333801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.344135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 452/893] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:34:06 2022] GET /identity/v3/endpoints?service_id=9e2cb3b260f34f2ca66b65154fe9572e&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:06.345100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.362792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c6489-41c3-4887-bd79-884e794ed416 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.407433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.408041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.408327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.408603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.417859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.419352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.425718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-9ab62228-9e99-48aa-a7b7-64ce5e92a846 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:06.431367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:06.432285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:06.433187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 442/894] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:34:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:06.437430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4455e-78db-4965-9f41-7b8f30882cd1 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.437757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4455e-78db-4965-9f41-7b8f30882cd1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.438029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b4455e-78db-4965-9f41-7b8f30882cd1 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.440173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 443/895] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:06.442322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4455e-78db-4965-9f41-7b8f30882cd1 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.473721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4382665-fa4b-47e5-a669-1df81094dc55 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.495819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:06.511343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24006f21-7941-4d43-ad84-ffd973095bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.511995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24006f21-7941-4d43-ad84-ffd973095bc8 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.512272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24006f21-7941-4d43-ad84-ffd973095bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.512541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24006f21-7941-4d43-ad84-ffd973095bc8 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.513156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24006f21-7941-4d43-ad84-ffd973095bc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.514573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24006f21-7941-4d43-ad84-ffd973095bc8 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.528976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 453/896] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:34:06 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:34:06.528976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.528976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.528976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.542272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:06.543676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:06.573680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b4455e-78db-4965-9f41-7b8f30882cd1 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.614300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.615167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.615517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.615887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.616767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:06.617137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.618831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.620287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 454/897] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:06.649594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 444/898] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 05:34:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:06.655813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.720859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:06.768737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2afc92-8084-4c94-8721-15a1ae53766d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:06.804313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3797a-11ce-427e-b778-6c4f3739c0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:06.804927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3797a-11ce-427e-b778-6c4f3739c0d3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:06.805233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3797a-11ce-427e-b778-6c4f3739c0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:06.805596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3797a-11ce-427e-b778-6c4f3739c0d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:06.880655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3797a-11ce-427e-b778-6c4f3739c0d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:06.882140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3797a-11ce-427e-b778-6c4f3739c0d3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:06.886498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:06.947532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.002211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a0bb6-625a-42ff-a178-991e938012b2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.003686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.005229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 455/899] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:06 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:07.006754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7dacf-f655-467c-aaf2-d68fa2a5abbe None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.007046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7dacf-f655-467c-aaf2-d68fa2a5abbe None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.007346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7dacf-f655-467c-aaf2-d68fa2a5abbe None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.008945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 456/900] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:07.014631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.014909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.015181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.033511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:07.034715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:07.047084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.048219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.048774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.049383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.050656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.052637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.070240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 445/901] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:06 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:34:07.071677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.129590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 457/902] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:07.216923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6777c2c-3a07-470d-96dd-43ac4cb70958 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.253100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.253706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.253992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.254266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.271433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.272901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:07.290185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:07.291159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:07.292008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:07.292008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:07.292008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:07.292008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 446/903] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:07.296808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.382683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.406818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62157996-bfc4-413a-b2f1-16455dfae7ce None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.410624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.428559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.449788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.450515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.450812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.451193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.451847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:07.452149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.453520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.469038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 458/904] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:07.474388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.528824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d170a26-c70a-4ccc-bcb9-bddfacacec63 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.564420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.565118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.565399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.565882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.566559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.568111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:07.579659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:07.581040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:07.581920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 447/905] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:07.588817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.606144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583349e0-94aa-4540-a264-e8edacfa4204 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.640432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.641050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.641331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.641627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.648810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.650820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:07.663194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:07.664527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:07.666122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:07.668665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 459/906] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:07.670889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.674065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.679570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.705786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03dd77-e4e8-4c66-938c-be1322abec1b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.742757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.743672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.744122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.744551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.745319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:07.745795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.747334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.752873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 448/907] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:07.760762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.784970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8400b54-3693-41b0-bfa1-44d9535675e5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.824462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.825076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.825356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.825723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.826413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:07.826868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.828476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.845044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 460/908] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:07.850663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.865538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea07ef-9d46-4e53-bfb9-4183deb7c269 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:07.903920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:07.904525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:07.904850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:07.905124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:07.905784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:07.906084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:07.907438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:07.919953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 449/909] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:07.920818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:07.930135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.937159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:07.985133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12834dee-503a-46bf-9a03-be0ffc4b8bff None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:08.019980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:08.020699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:08.021038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:08.021382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:08.022227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] RBAC: Adding query filter params (role.id=78d11a72db5a4392971caa71134bde31, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cca6991f4cff4e8b94c3d8b727fb6633) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:08.022601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:08.024805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:08.030883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 461/910] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/role_assignments?user.id=cca6991f4cff4e8b94c3d8b727fb6633&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=78d11a72db5a4392971caa71134bde31 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:08.059786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe235b-4253-46e9-8422-f6e2cef4932e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:08.098002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:08.098629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:08.098904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:08.099171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:08.099879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] RBAC: Adding query filter params (interface=public, service_id=22ac308c7def423382c152fb35487575, region_id=RegionOne) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:08.100182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:08.101590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bcc85-fc80-4411-baf9-629b8f12627c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:08.111362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 450/911] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:34:07 2022] GET /identity/v3/endpoints?service_id=22ac308c7def423382c152fb35487575&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:08.117080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:08.226582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035e164-2bd6-41bf-8943-e5434ab20f7e None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:08.255255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:08.269039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2956610d-c291-4ee0-8273-0211c8847923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:08.269788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956610d-c291-4ee0-8273-0211c8847923 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:08.270323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956610d-c291-4ee0-8273-0211c8847923 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:08.271147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2956610d-c291-4ee0-8273-0211c8847923 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:08.271547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956610d-c291-4ee0-8273-0211c8847923 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:08.272934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2956610d-c291-4ee0-8273-0211c8847923 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:08.280027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 462/912] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:34:08 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:34:08.659028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decc77c-707d-4646-b354-70d84e446636 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:08.659747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decc77c-707d-4646-b354-70d84e446636 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:08.659747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5decc77c-707d-4646-b354-70d84e446636 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:08.661133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 451/913] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:08.667725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:08.667725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:08.667725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:08.687347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:08.688888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:08.760128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 463/914] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:08 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:09.026477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decc77c-707d-4646-b354-70d84e446636 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:09.100079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:09.134941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5decc77c-707d-4646-b354-70d84e446636 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:09.171415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:09.172401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.172896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.173371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.174175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:09.175872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-1822541d-4181-4ca9-8c0b-0111416398b5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:09.185792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:09.186861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 05:34:09.187777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:09.190578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 452/915] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:09 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:09.197232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:09.233665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95b951-38b2-4f13-bfc1-89752eabe9fc None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.235820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95b951-38b2-4f13-bfc1-89752eabe9fc None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.236322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95b951-38b2-4f13-bfc1-89752eabe9fc None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.238093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 453/916] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:09.246488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.247311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.247798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.279940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:09.285621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:09.317315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51396cc0-4e76-4026-9b6c-d6944b3d8d00 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:09.340808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:09.360532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:09.361762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.362651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.363421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.364512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:09.365428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:09.367463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:09.373973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 464/918] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:09 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:09.377621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 454/918] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:09.378954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:09.490228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64362ec7-cb66-46b1-83b6-a29f0782c813 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:09.526982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:09.527591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.527867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.528138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.528761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:09.529062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:09.530565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:09.535853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 465/919] 10.222.0.73 () {56 vars in 1114 bytes} [Wed Aug 24 05:34:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:09.541143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:09.606265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:09.645819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:09.646796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c917e-b73b-4fe6-a41e-daa03ab6e097 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:09.686515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:09.687922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.688425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.689423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.690283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:09.692740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:09.698986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:09.721253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 455/920] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:34:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:09.766332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f28e5-1bc1-49c1-a6d0-075fff62f186 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:09.813720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:09.814288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.814565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.814835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.815418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:09.816936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:09.827388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:09.828433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:09.829282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 466/921] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:09.834573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:09.946545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745f79a7-a232-4a63-bc39-80876dbe4473 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:09.972579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:09.984099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:09.984710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:09.984989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:09.985264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:09.985928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:09.987903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:09.987903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df4df0c-ac9e-4ce4-a6f0-db3c121b46b0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:09.992730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 456/922] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:09.998272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.061943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce8a8b-bb5c-4ac7-a8af-077e54cd045c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.062290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce8a8b-bb5c-4ac7-a8af-077e54cd045c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.062499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ce8a8b-bb5c-4ac7-a8af-077e54cd045c None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.064169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 457/923] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:10.069024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa45e1-cd81-48b1-bcb5-97b36a077b08 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.069304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa45e1-cd81-48b1-bcb5-97b36a077b08 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.069587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fa45e1-cd81-48b1-bcb5-97b36a077b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.089989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d7fa45e1-cd81-48b1-bcb5-97b36a077b08 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:10.091261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d7fa45e1-cd81-48b1-bcb5-97b36a077b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:10.104261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d636d7f-3ea4-4c36-a19b-e160efe3aa07 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:10.122988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:10.140824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:10.141842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.142318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.142782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.143703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:10.144221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:10.145816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:10.151357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 467/924] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:10.157505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.172587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 458/925] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:10.270874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac3ed42-9183-4c0b-9148-8dcffc5b6262 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:10.287126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9968-05ff-40c8-ba5b-b09fd9e2e8dc None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.287424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9968-05ff-40c8-ba5b-b09fd9e2e8dc None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.287674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047b9968-05ff-40c8-ba5b-b09fd9e2e8dc None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.289199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 459/926] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:10.295358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.295642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.295914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.312562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:10.313169 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.313450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.313748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.314604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] RBAC: Adding query filter params (interface=public, service_id=219b7d21cb09416a80fa2b4d8c05bf25, region_id=RegionOne) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:10.314856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:10.316183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:10.316305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:10.317376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:10.321826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 468/927] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/endpoints?service_id=219b7d21cb09416a80fa2b4d8c05bf25&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:10.330531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.369732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:10.405716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 460/928] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:10.439967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.445986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9a1cc-81bb-4bd0-bd4a-69e207da83c3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:10.491104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:10.491729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.492012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.493118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.493538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:10.494979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:10.501734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 469/929] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/services => generated 1497 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:34:10.519255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:10.602913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965302-a9a2-4a0e-9cc4-a9dcb14d7de2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:10.643122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:10.644145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.644644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.645123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.656135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:10.660937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:10.661969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:10.662796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 461/930] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:10.668075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.699594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.796658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:10.800932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fb44c-9d49-4652-a83d-4d3c0d3f2950 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:10.817104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abdcb05-01da-47f3-b3b9-32adf160de6e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:10.819547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:10.842555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:10.843565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.844109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.844594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.845464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:10.846027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:10.847656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:10.857007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:10.857587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:10.857897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:10.858241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:10.858727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 470/931] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:10.861814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:10.863700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:10.864439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:10.876709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:10.877722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 05:34:10.878575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:10.879284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 462/932] 10.222.0.73 () {56 vars in 1132 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:10.885620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:10.991267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e12f51c-8ca2-4fb6-817b-f28003b6ef68 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.023283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a694ca9-4aa0-4b50-913d-5cc5d0676098 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.033363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.033777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.034142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.034483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.045837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.047383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:11.062705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:11.063930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:11.065011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:11.065011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:11.065011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:11.068724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.069515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.069660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 471/933] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:11.070132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.070558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.071416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.072012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:11.072457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.074010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.085703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 463/934] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:11.086373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.099116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:11.124453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:11.197755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfdc39f-8ab3-4317-b96f-6adaf66a2d11 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.201115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500aa5f-d509-4558-9e73-12eb7a0b03b1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.236031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.237010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.237435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.237892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.238596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.238969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:11.239397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.239556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.239854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.240224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.240950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:11.241428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.241719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.243676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.254070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 464/935] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:11 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:11.255096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.275330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 472/936] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:11.283583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.359549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:11.366943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:11.374666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a25968-0084-4172-9537-0b1452262e90 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.389907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af80794-5625-48df-ab05-549da88fe1c1 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.412042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.413020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.413482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.413988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.415477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.417060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.427870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.428533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.428831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.429138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.438026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.439466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.441003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 465/937] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:34:11 2022] POST /identity/v3/endpoints => generated 347 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:11.451984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:11.453004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:11.453988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:11.459406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 473/938] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:34:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:11.466370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.596876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf19ce4-5d8e-4216-ab29-0c1c91871b0a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.634756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.635298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.635581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.635857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.636601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:11.636908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.638455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.657586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 466/939] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 05:34:11 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:11.663174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.778627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea4a410-bcc0-418f-9c5d-b567aaea46ea None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:11.820493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:11.821164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.821412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.821742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.824043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:11.833886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:11.835364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-f02dd40d-d13d-427f-bf57-0479a5d619a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:11.849725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:11.850859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:11.851729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:11.851729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:11.851729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:11.851729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 474/940] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:11.857616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:11.867012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:11.877468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdca21-bc5c-4719-bbd3-5ceff3c5ff36 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.877805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdca21-bc5c-4719-bbd3-5ceff3c5ff36 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.878074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdca21-bc5c-4719-bbd3-5ceff3c5ff36 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.879689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 475/941] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:11.893460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:11.894297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:11.894574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:11.915209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:11.916460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:11.982093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dbfb1d-0d43-43ef-8a03-c7141bf2e75e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.005636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 476/942] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:12.020955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.022013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.022520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.023044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.024575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:12.025079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.026724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.044608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 467/943] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:12.046268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:12.154783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95ac97-4f8e-4847-a3b8-0596faa5fad6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.172535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:12.191553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.192492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.192994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.193509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.201350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.203354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:12.215517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:12.216436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:12.217311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.224095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 477/944] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:12.225674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:12.277588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:12.299894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:12.339480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d37bf-f73c-4dd1-b256-b4683ca2dc68 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.377389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.378159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.378532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.378899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.379684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:12.380085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.381632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.395545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 468/945] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:12.403364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:12.411129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30349218-a93d-4f10-9a0c-33dfbd365a04 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.454295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.455055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.455340 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.455609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.456250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.458273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.462163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING keystone.server.flask.common [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 05:34:12.488708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.sql.core [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:34:12.488708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 05:34:12.488708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [parameters: {'id': '759f0a1760534c54bc2ea681ce0528f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 05:34:12.488708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 759f0a1760534c54bc2ea681ce0528f8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 759f0a1760534c54bc2ea681ce0528f8. Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:34:12.498990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result.read() Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 05:34:12.500217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application yield resource Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise self.value Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application session.commit() Aug 24 05:34:12.501522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:34:12.502666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:34:12.503765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result.read() Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:34:12.504760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [parameters: {'id': '759f0a1760534c54bc2ea681ce0528f8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:12.514546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:12.515531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 759f0a1760534c54bc2ea681ce0528f8. Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 478/946] 10.222.0.73 () {60 vars in 1140 bytes} [Wed Aug 24 05:34:12 2022] POST /identity/v3/projects => generated 262 bytes in 227 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:12.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:12.533001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175407fc-517e-40aa-9491-fb295ea3ca92 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.553225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:12.564925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:12.577311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4e54c-56f2-4023-8a9c-576af264b407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.578270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e54c-56f2-4023-8a9c-576af264b407 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.578631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e54c-56f2-4023-8a9c-576af264b407 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.578993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e54c-56f2-4023-8a9c-576af264b407 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/cca6991f4cff4e8b94c3d8b727fb6633/roles/78d11a72db5a4392971caa71134bde31` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.581692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4e54c-56f2-4023-8a9c-576af264b407 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=cca6991f4cff4e8b94c3d8b727fb6633, role_id=78d11a72db5a4392971caa71134bde31)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.583581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4e54c-56f2-4023-8a9c-576af264b407 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.601819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 469/947] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:34:12 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/cca6991f4cff4e8b94c3d8b727fb6633/roles/78d11a72db5a4392971caa71134bde31 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:34:12.607090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894fd8a-94ed-4aed-b1c6-cdc87680ac64 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.607090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894fd8a-94ed-4aed-b1c6-cdc87680ac64 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.607090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894fd8a-94ed-4aed-b1c6-cdc87680ac64 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.607090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 470/948] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:12.613118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.613409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.613887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.636161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:12.637741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:12.654615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd4d96-79ee-488b-a8e2-92ab9bf076d4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.696693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.697871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.698292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.698691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.707071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.709103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:12.724165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:12.725350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:12.726469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:12.729191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 479/949] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:12.734980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:12.740081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 471/950] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:12.845838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:12.856049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac8e0b1-a8b1-4145-ba1c-7e01153059af None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:12.891911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:12.892532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:12.892808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:12.893084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:12.893772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:12.894081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:12.895503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:12.917723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 480/951] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:13.020274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:13.136776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482d1d9-debf-41fe-b896-c2dd03a685f1 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:13.155300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:13.177142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:13.177814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:13.178098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:13.178368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:13.178947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:13.180363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:13.189600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:13.190569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:13.191388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 472/952] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:13.199221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:13.312441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de992408-62a9-4d2c-9e51-c109671b8946 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:13.349716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:13.350329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:13.350610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:13.350881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:13.351520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:13.351840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:13.353203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1ea9d5-8a33-4e38-8f93-b3d7ad7672e4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:13.358840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 481/953] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:13.366843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:13.469746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0339-4316-49d3-893b-635f09b66e2c None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:13.470097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0339-4316-49d3-893b-635f09b66e2c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:13.470305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c0339-4316-49d3-893b-635f09b66e2c None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:13.471897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 482/954] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:13.476541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:13.476855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:13.477135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:13.477249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f07c96e-f36c-4fc0-b9a8-24996de374f5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:13.501975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:13.504211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:13.514926 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:13.515533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:13.515810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:13.516140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:13.516772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:13.517119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:13.518524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:13.519084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:13.524470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 473/955] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:13.530102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:13.580997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:13.601635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 483/956] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:13.653659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4f383-9e36-41cf-bea0-44f296ff648b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:13.688344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:13.689007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:13.689290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:13.689582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:13.690241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:13.691610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:13.715438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 474/957] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:34:13 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:34:13.890773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:13.990947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6f349-744e-4245-a958-8c5390f10618 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:14.032011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:14.032705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:14.032991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:14.033265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] PATH_INFO: `/v3/services/compute` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:14.033872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:14.035220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:14.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:14.044992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:14.046503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 484/958] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:14.051342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:14.156354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:14.162134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339e6c9-40d6-428a-93f8-597ff14ee754 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:14.199235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:14.199852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:14.200131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:14.200400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:14.201037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:14.201320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:14.202810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896fb247-f30a-4ceb-a8a8-13e72fa1a041 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:14.208205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 475/959] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:14.214692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:14.224940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:14.332922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1df86-a2dd-43b9-940b-2cd37094039a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:14.368467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:14.369070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:14.369346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:14.369637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:14.370264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:14.370560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:14.371926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b7fea4-2772-4dff-9d67-4f02361b4913 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:14.376970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 485/960] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:14 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:14.696687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ae8d3-be7e-4aec-b21b-30e39a0c18bf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:14.697131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ae8d3-be7e-4aec-b21b-30e39a0c18bf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:14.697502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ae8d3-be7e-4aec-b21b-30e39a0c18bf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:14.699455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 476/961] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:14.704938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3313c-c087-445c-85e9-79cd915ad394 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:14.705231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3313c-c087-445c-85e9-79cd915ad394 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:14.705507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf3313c-c087-445c-85e9-79cd915ad394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:14.725293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-bcf3313c-c087-445c-85e9-79cd915ad394 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:14.726543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bcf3313c-c087-445c-85e9-79cd915ad394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:14.793745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:14.802451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 486/962] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:15.074490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ae8d3-be7e-4aec-b21b-30e39a0c18bf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.097054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c052720-64e3-4904-9949-b07f7648b5b7 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.097912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c052720-64e3-4904-9949-b07f7648b5b7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.098402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c052720-64e3-4904-9949-b07f7648b5b7 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.100082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 487/963] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:15.105779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.106320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.106800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.126658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:15.127867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:15.182105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ae8d3-be7e-4aec-b21b-30e39a0c18bf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.209586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 488/964] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:15.218130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.218817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.219097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.219373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.231202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:15.236036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:15.236934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:15.238167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 477/965] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:15.243954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.349739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f8368-deb3-4de3-bf4d-ffff20a54af7 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.381602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:15.387825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.388442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.388721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.389000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.389687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:15.389993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:15.391354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:15.400833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 489/966] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:15.406551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.434792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:15.499956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.523315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc8e49d-ad57-43df-a368-14e9ecc97dad None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.571730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.572384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.572659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.572933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.586166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:15.588702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:15.608254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:15.609202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:15.610822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:15.610822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:15.610822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:15.610822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 478/967] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:15.615206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.635122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df83645c-c608-4698-9b72-0b1ebcd20a87 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.674955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.675567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.675846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.676168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.676755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:15.678258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:15.694088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.sql.core [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:34:15.694088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 05:34:15.694088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [parameters: {'user_id': '94cedaa996cc4c3c92059fa5c3eefc9b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 05:34:15.694088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88146) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result.read() Aug 24 05:34:15.702574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:34:15.703445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application yield resource Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application next(self.gen) Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise self.value Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application session.commit() Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 05:34:15.709723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.session.flush() Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application compat.raise_( Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 05:34:15.710617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 05:34:15.712032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application conn.query(q) Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application result.read() Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 05:34:15.712854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [parameters: {'user_id': '94cedaa996cc4c3c92059fa5c3eefc9b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:15.714622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 05:34:15.716825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 490/968] 10.222.0.73 () {60 vars in 1131 bytes} [Wed Aug 24 05:34:15 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:15.719078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.722577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:15.749328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16b0aa-f51d-4962-8835-78315be79363 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.777395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:15.787591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.788411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.788836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.789272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.790614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:15.791061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:15.792588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dfefe3-2933-4100-a0e4-f38399b273ba None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:15.809392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 479/969] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:15.811008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b149062-d1b0-40bf-8e56-7cb5e0cc179d None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.811476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b149062-d1b0-40bf-8e56-7cb5e0cc179d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.811904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b149062-d1b0-40bf-8e56-7cb5e0cc179d None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.813226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 480/970] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:15.813943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b149062-d1b0-40bf-8e56-7cb5e0cc179d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:15.832066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac97e46-09e7-4fe1-baa7-fd01e40a3370 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.873368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.873950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.874228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.874495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.881964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:15.883635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-eaab86e1-1bc8-47a6-89aa-45b64267c88c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:15.897586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:15.898576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 05:34:15.900066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:15.904164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 491/971] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:15.905732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.906024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.906298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.925120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:15.926994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:15.929417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b149062-d1b0-40bf-8e56-7cb5e0cc179d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:15.967443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:15.968081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:15.968481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:15.968755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:15.971927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:15.978526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:15.980862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:15.994071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:15.995039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 481/972] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:15.995960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.003555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 492/973] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:16.004283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.058589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.114797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32f39f-f8d2-448b-957e-909d3dfea2cc None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.119674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73f91e2-8546-4746-b0a5-16ea1829733a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.150860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.151380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.151658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.151931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.152627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:16.152929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.154405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.156670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.157403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.157787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.158161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.158968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:16.159342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.160823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.189837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 493/974] 10.222.0.73 () {56 vars in 1149 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:16.191136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.201728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 482/975] 10.222.0.73 () {56 vars in 1101 bytes} [Wed Aug 24 05:34:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:16.282939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.313375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8bfea-1288-48ac-bfe2-9936dc1c5534 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.315689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.349606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.362789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a4c967-d914-4a74-859c-6631b079cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.363435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c967-d914-4a74-859c-6631b079cd4d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.363721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c967-d914-4a74-859c-6631b079cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.363999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a4c967-d914-4a74-859c-6631b079cd4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.396620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85378dae-e819-4bd9-baae-d47809587c0c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.440823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.441390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.441769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.442053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.442641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.443598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4c967-d914-4a74-859c-6631b079cd4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.444016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.445199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a4c967-d914-4a74-859c-6631b079cd4d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-67f5fa59-2999-40aa-8d0c-00b33eef58d6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:16.453393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:16.454463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:16.455288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 483/976] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:16.461033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1c903-f92a-4b55-bf11-e34a103b65da None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.461890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1c903-f92a-4b55-bf11-e34a103b65da None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.462172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1c903-f92a-4b55-bf11-e34a103b65da None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.468532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 484/977] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:16.468532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1c903-f92a-4b55-bf11-e34a103b65da None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.557318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 494/978] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:16.558830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.559119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.559390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.565704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.576127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1c903-f92a-4b55-bf11-e34a103b65da None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.581225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:16.582433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:16.593002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.610100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.610702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.610983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.611254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.611884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:16.612185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.613549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.623736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 485/979] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:16.623736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.652933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 495/980] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:16.653804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.740925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb539b9-6e23-42ec-91fa-6d87c6e3ae4b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.757846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d76c4-9738-4ec5-8879-b3aaf8bd2324 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.778369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.778995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.779276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.779552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.787455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.792669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.794098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.794224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.794732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.794993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.795267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.795899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:16.796200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.797554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.802919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 496/981] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:16.808449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:16.811749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:16.812652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:16.813515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:16.813515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:16.813515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:16.817485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 486/982] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:16.818374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.855688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:16.920892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af4fb4-61a5-4573-8e5d-3af989d6a8a7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.924201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012ce00a-53c2-4080-84e5-a65b02a7b570 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:16.954533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.955134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.955409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.955676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.956374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] RBAC: Adding query filter params (interface=admin, service_id=219b7d21cb09416a80fa2b4d8c05bf25, region_id=RegionOne) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:16.956687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.958076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.958373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:16.958975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:16.959262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:16.959536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:16.960174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:16.960479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:16.961961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:16.963372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 497/983] 10.222.0.73 () {56 vars in 1249 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/endpoints?service_id=219b7d21cb09416a80fa2b4d8c05bf25&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:16.964145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:16.978519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 487/984] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:16.979160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:17.025701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:17.068753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b93e54-e0d5-4a92-96b3-348158a206ff None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:17.073754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:17.081431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6b3591-a5f7-49b6-8868-9b2f7c4a54ad None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:17.104395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:17.105011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:17.105291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:17.105581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:17.106228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:17.107590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:17.116623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:17.117260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:17.118164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:17.118446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:17.119046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:17.120409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:17.123187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 498/985] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:16 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:17.123994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:17.127494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 488/986] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:34:16 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:34:17.268764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e4469-02f4-4995-adba-8e89c3e90b71 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:17.288105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:17.310813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:17.311459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:17.311738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:17.312009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:17.319150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:17.321271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:17.332644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:17.333799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:17.334828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 499/987] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:17.340202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:17.451398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51c25e-aa4c-4320-a192-e679e1287542 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:17.488657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:17.489268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:17.489546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:17.489845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:17.490536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:17.490841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:17.492262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5011f1d7-11c3-49db-b4e8-ab557b0f2d8c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:17.506130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 489/988] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:17.512791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:17.622007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ae9650-2578-4b9c-85d7-f7801e946a75 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:17.655961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:17.656567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:17.656844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:17.657115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:17.657868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] RBAC: Adding query filter params (role.id=78d11a72db5a4392971caa71134bde31, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=cca6991f4cff4e8b94c3d8b727fb6633) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:17.658177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:17.659712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdba552-249d-4dbe-ad44-591fffb0017e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:17.665851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 500/989] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:17 2022] GET /identity/v3/role_assignments?user.id=cca6991f4cff4e8b94c3d8b727fb6633&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=78d11a72db5a4392971caa71134bde31 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:19.079649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b0027-559a-4d75-9568-d90cd4ba51ca None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.080338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b0027-559a-4d75-9568-d90cd4ba51ca None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.080338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b0027-559a-4d75-9568-d90cd4ba51ca None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.081623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 490/990] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:19.086983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.087471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.087647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.105931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:19.107115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:19.113128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f3213-e28a-4689-b60d-f2c4e98a223b None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.113409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f3213-e28a-4689-b60d-f2c4e98a223b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.113710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7f3213-e28a-4689-b60d-f2c4e98a223b None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.115305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 491/991] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:19.120705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.120984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.121255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.138730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:19.139901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:19.151234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:19.154822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:19.196633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 501/992] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:19.208809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 492/993] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:19.465424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:19.481049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:19.575849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3454a3-1a6f-4228-9caf-0e572c43566d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:19.588749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07824fc5-bdfe-4a77-802c-71fbc68e4c67 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:19.612366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:19.612995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.613267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.613536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.614135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:19.615490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:19.623321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:19.623944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.624220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.624487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.625055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-9a7296b4-1658-4469-a8ca-3cbf2947f774 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:19.625369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:19.626552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:19.627445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:19.628172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:19.631460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 502/994] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:19.632611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8aac-0a16-41cf-8eee-49a7542740e6 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.632958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8aac-0a16-41cf-8eee-49a7542740e6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.633746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e8aac-0a16-41cf-8eee-49a7542740e6 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.635105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 503/995] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-9324a711-c070-42b5-bce7-479d16d7b15e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:19.635775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:19.640864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 05:34:19.643403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:19.644384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8aac-0a16-41cf-8eee-49a7542740e6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:19.644473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 493/996] 10.222.0.73 () {56 vars in 1126 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:19.644473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.644473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.644473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.658411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:19.659620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:19.667134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:19.675456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:19.727807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 494/997] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:19.728867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:19.747586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e8aac-0a16-41cf-8eee-49a7542740e6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:19.783365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:19.783978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.784259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.784532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.785399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:19.785604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:19.787257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:19.793112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 504/998] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:19.798504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:19.844807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da41001-74c4-4d9f-9d65-91e121671c0e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:19.883769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:19.884376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.884655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.884917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.885535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:19.885876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:19.887237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:19.893245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 495/999] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:19.899374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:19.908425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:19.912228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:19.917641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753cfc4-131d-4830-8e79-9955e23cae35 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:19.956259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:19.957126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:19.957600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:19.957707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:19.958339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:19.958637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:19.959988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:19.970183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 505/1000] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:19.971175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:20.016971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c906b9-dd78-4b11-8409-bf096dec6b77 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.050723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.051321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.051597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.051866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.052484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:20.052784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.054173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.059579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 496/1001] 10.222.0.73 () {56 vars in 1122 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:20.060350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:20.076939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea68a6-c298-45c1-83b3-258962f897a3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.110519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.111154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.111382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.111643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.112322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] RBAC: Adding query filter params (interface=public, service_id=8531ca59a57e426790c2ec8862fb2892, region_id=RegionOne) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:20.112624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.113989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.119609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 506/1002] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:34:19 2022] GET /identity/v3/endpoints?service_id=8531ca59a57e426790c2ec8862fb2892&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:20.120352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:20.129678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:20.149296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:20.162143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253280ca-910e-4d4c-8866-3019ea263b3f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.195472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.196092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.196366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.196634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] PATH_INFO: `/v3/services/image` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.197202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.198585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:20.207011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:20.208191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:20.209024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 497/1003] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 05:34:20 2022] GET /identity/v3/services/image => generated 86 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:20.209846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:20.222537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c4cf8-02ad-4892-af9c-2ba0e6d05319 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.257356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.258038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.258325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.258627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.259294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.260661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.279068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 507/1004] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:34:20 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:20.279947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:20.326694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e7eaea-0260-4f3a-bd03-9eef6a40337f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.371386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:20.380234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.380856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.381590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.381855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.382746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.384410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.385142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:20.405621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 498/1005] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:34:20 2022] GET /identity/v3/services => generated 1727 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:34:20.410106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7114ac-39e9-468a-a709-09720244cd5f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.454098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.454825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.455567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.455567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.456002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:20.456304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.457718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c278c-44c2-4f44-9777-f7f157d924e6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.464052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 508/1006] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:20.470001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:20.590635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8eafb-085c-4672-abac-5dfdb0e13765 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:20.627332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:20.627951 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:20.628220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:20.628488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:20.629118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] RBAC: Adding query filter params (type=image) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:20.629416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:20.630913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdea5de-5f82-4c22-9489-2e488c231906 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:20.637488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 499/1007] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:20 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:22.240261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746c127-2416-4def-8e81-cf6e2b34abda None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.240780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746c127-2416-4def-8e81-cf6e2b34abda None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.240780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746c127-2416-4def-8e81-cf6e2b34abda None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.242876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 509/1008] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:22.248400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1921a-3f01-40dd-a634-e6ec98af35ed None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.248851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1921a-3f01-40dd-a634-e6ec98af35ed None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.249022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d1921a-3f01-40dd-a634-e6ec98af35ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.273731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-29d1921a-3f01-40dd-a634-e6ec98af35ed None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:22.274940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-29d1921a-3f01-40dd-a634-e6ec98af35ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:22.331648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:22.346543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 500/1009] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:22.422915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8ca5-2df2-4f0b-bf36-46adddd37279 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.423260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8ca5-2df2-4f0b-bf36-46adddd37279 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.423463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779b8ca5-2df2-4f0b-bf36-46adddd37279 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.424921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 510/1010] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:22.430500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6bd5-a12d-4246-96f0-a50b29496307 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.430835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6bd5-a12d-4246-96f0-a50b29496307 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.431047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e6bd5-a12d-4246-96f0-a50b29496307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.449028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ef5e6bd5-a12d-4246-96f0-a50b29496307 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:22.450232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ef5e6bd5-a12d-4246-96f0-a50b29496307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:22.527063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 501/1011] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:22.557636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f91f16-4507-461b-add6-44483374ccce None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.557950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f91f16-4507-461b-add6-44483374ccce None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.558194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f91f16-4507-461b-add6-44483374ccce None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.559656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 511/1012] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:22.564801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.565094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.565369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.583244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:22.584431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:22.617429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f91f16-4507-461b-add6-44483374ccce None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:22.661111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 502/1013] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:22 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:22.729423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f91f16-4507-461b-add6-44483374ccce None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:22.763036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:22.763959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.764465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.764938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.766254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:22.767875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:22.770045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 512/1014] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:22.776025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:22.797193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:22.817477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:22.830400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:22.883961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb4c12a-8cb2-4b33-8507-5c3f6b60ce2b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:22.909097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ad130-6dd3-4149-a3ba-edf64d879be7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:22.920264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:22.920861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.921136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.921406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.922082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:22.923512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:22.948244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:22.948875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:22.949157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:22.949428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:22.950043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:22.951400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:22.951906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 503/1015] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:22 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:22.952719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:22.960669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:22.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:22.963473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 513/1016] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:22.968144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:23.067297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:23.067990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:23.072503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734ed1b-f27f-4768-a95e-339b1beee454 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:23.093227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef269a5-2180-4cac-94cd-d2e5573c63d4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:23.115032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:23.115653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:23.117143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:23.117143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:23.117143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:23.118443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:23.134219 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:23.134834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:23.135114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:23.135412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:23.136049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:23.136352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:23.137794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:23.137979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 504/1017] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:22 2022] POST /identity/v3/services => generated 242 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:23.143202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 514/1018] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:23.149540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:23.281374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e1056-e2dd-4744-8776-e11463831880 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:23.318903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:23.319525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:23.319805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:23.320077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:23.320744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] RBAC: Adding query filter params (type=compute) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:23.321051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:23.322466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07582747-33c2-4cca-bbd6-53eb8c2b62bb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:23.328189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 505/1019] 10.222.0.73 () {56 vars in 1112 bytes} [Wed Aug 24 05:34:23 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:23.333819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:23.438925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24dc555-1c60-4240-ad58-99b436bb5763 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:23.473793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd13a68a-37a0-4ec9-8bbd-e34ea447d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:23.474403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a68a-37a0-4ec9-8bbd-e34ea447d2a1 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:23.474681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a68a-37a0-4ec9-8bbd-e34ea447d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:23.474949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd13a68a-37a0-4ec9-8bbd-e34ea447d2a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:23.475614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a68a-37a0-4ec9-8bbd-e34ea447d2a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:23.477556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd13a68a-37a0-4ec9-8bbd-e34ea447d2a1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:23.505701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 515/1020] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:34:23 2022] POST /identity/v3/endpoints => generated 334 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:34:24.935333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b1cbf-d7a1-4c10-99e8-9f4337a65f2a None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:24.935882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b1cbf-d7a1-4c10-99e8-9f4337a65f2a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:24.935882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b1cbf-d7a1-4c10-99e8-9f4337a65f2a None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:24.937350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 506/1021] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:24.942403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b9a4-1347-4ffe-97fd-524e11305c35 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:24.942797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b9a4-1347-4ffe-97fd-524e11305c35 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:24.942960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427b9a4-1347-4ffe-97fd-524e11305c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:24.949705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:24.962336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b427b9a4-1347-4ffe-97fd-524e11305c35 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:24.963521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b427b9a4-1347-4ffe-97fd-524e11305c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:25.039709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 516/1022] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:25.095296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122c877-43a1-4cc9-b51b-deb4a54b3b7a None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.095629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122c877-43a1-4cc9-b51b-deb4a54b3b7a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.095859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122c877-43a1-4cc9-b51b-deb4a54b3b7a None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.097330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 507/1023] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:25.103066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.103351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.103883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.122510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:25.123699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:25.191974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 517/1024] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:25.303298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122c877-43a1-4cc9-b51b-deb4a54b3b7a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:25.325649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:25.408995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122c877-43a1-4cc9-b51b-deb4a54b3b7a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:25.444975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:25.445550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.445850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.446121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.447236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:25.448663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:25.450257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 508/1025] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:25.456176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:25.467063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:25.525188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:25.559905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:25.569554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90ebf1-c656-4fca-b149-1b9d24466f7b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:25.572841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a3203a-543e-477d-b9f8-45f9191c92f5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:25.604026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a2561-b44c-49ed-babd-1d14beed30b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:25.604639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2561-b44c-49ed-babd-1d14beed30b0 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.604917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2561-b44c-49ed-babd-1d14beed30b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.605187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a2561-b44c-49ed-babd-1d14beed30b0 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.605857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a2561-b44c-49ed-babd-1d14beed30b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:25.606276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:25.606899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.607176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.607274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a2561-b44c-49ed-babd-1d14beed30b0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:25.607451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] PATH_INFO: `/v3/services/image` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.608024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:25.609362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e78df010-16b3-4aad-b3df-9ed132e71426 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:25.618596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:25.619660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:25.620614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 509/1026] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 05:34:25 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:25.621427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f8eec-7fc3-4a71-9779-64978f30c738 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.621427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f8eec-7fc3-4a71-9779-64978f30c738 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.621548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f8eec-7fc3-4a71-9779-64978f30c738 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.622887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 510/1027] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:25.626618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f8eec-7fc3-4a71-9779-64978f30c738 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:25.634657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 518/1028] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:34:25 2022] POST /identity/v3/users => generated 285 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:34:25.635891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.636185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.636459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.666352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:25.668265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:25.745321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 519/1029] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:25.747554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f8eec-7fc3-4a71-9779-64978f30c738 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:25.789733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:25.789733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.790055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.790385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.791231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] RBAC: Adding query filter params (name=image) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:25.791531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:25.792893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:25.798693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 511/1030] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:25.805361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:25.859809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:25.922828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c6db5-5594-40db-b040-2fea3ab35050 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:25.957360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:25.958043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:25.958287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:25.958519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:25.959143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] RBAC: Adding query filter params (type=image) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:25.959444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:25.960823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:25.965928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 520/1031] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:25.971859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:25.991238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:26.016264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.083795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54969f-929e-438b-aab6-b1cee2fcbe2e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.121662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.122271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.122593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.122724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0fe61d-32bd-412d-a494-4c2cbe6d1780 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.122865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.123557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] RBAC: Adding query filter params (interface=public, service_id=b13fd309ca134e78a0124f829099531a, region_id=RegionOne) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:26.123863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:26.125680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:26.131849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 512/1032] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:34:25 2022] GET /identity/v3/endpoints?service_id=b13fd309ca134e78a0124f829099531a&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:26.132613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:26.137168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.163563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.164716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.164989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.165796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.188058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:26.193469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:26.194480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:26.195357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:26.199578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 521/1033] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:26.200457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.229665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:26.251003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f78ffe-f93f-43ee-ac44-84fdf4671060 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.291225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.291855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.292106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.292442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.292974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:26.294387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:26.300916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 513/1034] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:34:26.310785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4661e4-f5f1-4b34-a3e1-be1f497589f5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.349498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362b25-604c-448a-8733-306715603e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.350158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362b25-604c-448a-8733-306715603e4b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.350442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362b25-604c-448a-8733-306715603e4b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.350712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45362b25-604c-448a-8733-306715603e4b None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.351427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362b25-604c-448a-8733-306715603e4b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:26.352280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362b25-604c-448a-8733-306715603e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:26.353974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45362b25-604c-448a-8733-306715603e4b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:26.368189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 522/1035] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:26.369482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.493548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d1cfc-73b1-4eba-a009-0b55de8ad9bb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.534653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.535279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.535560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.535831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.547207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:26.548649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:26.551180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:26.564855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:26.565971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:26.566920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:26.566920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:26.566920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:26.566920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 514/1036] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:26.573725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362b25-604c-448a-8733-306715603e4b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.605190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:26.676369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45362b25-604c-448a-8733-306715603e4b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.709793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.710401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.710679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.711504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.712148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:26.712450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:26.713836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:26.731152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 523/1037] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:26.732392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.836623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2cfae-6baa-4ac6-ae7d-ef2c785f8113 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:26.871806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:26.872439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:26.872716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:26.872987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:26.880700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:26.882167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:26.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:26.894381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:26.895329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 515/1038] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:26.902655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:26.995088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:27.006278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9a8bb9-955e-40cc-95e4-4cbd6bf516a5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:27.040800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:27.041407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.041716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.041983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.042705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:27.043009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:27.044418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:27.063571 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 524/1039] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:34:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:27.069132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:27.128699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:27.182518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74e951-5cb5-4fa3-b1e1-8a23f4ab340a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:27.219675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:27.220281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.220559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.220830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.297604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:27.299392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:27.355301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:27.400913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 516/1040] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:27 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:27.406804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:27.516023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66cda4-bf45-4651-a6b3-b00861894bc0 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:27.550829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:27.551462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.551740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.552011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.562146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:27.563590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:27.572589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-3ab2faec-a269-4af8-8246-3ce9dce1993b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:27.580231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:27.581171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:27.582074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:27.582074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:27.582074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:27.582074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 525/1041] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:27.588515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:27.662016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb14f4-b67f-4610-850b-f934000ef2ea None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.662205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb14f4-b67f-4610-850b-f934000ef2ea None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.662480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb14f4-b67f-4610-850b-f934000ef2ea None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.665592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 526/1042] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:27.669309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.669609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.669891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.688869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:27.690121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:27.695701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f5aa9-bf69-413a-a02b-7ebb3f23be00 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:27.734908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:27.735610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.735888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.736167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.736811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:27.737120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:27.738518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:27.754682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 517/1043] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:27.756694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:27.769596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 527/1044] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:27.776952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:27.862081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d35bc8-74a0-4251-a9eb-3adba8c275d0 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:27.895903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65004aac-76f5-4458-b156-744944b1167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:27.896527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65004aac-76f5-4458-b156-744944b1167d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:27.896805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65004aac-76f5-4458-b156-744944b1167d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:27.897074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65004aac-76f5-4458-b156-744944b1167d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:27.904491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65004aac-76f5-4458-b156-744944b1167d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:27.905967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65004aac-76f5-4458-b156-744944b1167d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-65004aac-76f5-4458-b156-744944b1167d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:27.918047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:27.918942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:27.924736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 518/1045] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:27.926964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.024877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0264b3-f760-489b-a0f1-0ef4c89d43a6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.044687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65004aac-76f5-4458-b156-744944b1167d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.054868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.061914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.062511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.062768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.063037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.063728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:28.064038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.065461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.082713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 528/1046] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:28.084613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.134427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.159817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65004aac-76f5-4458-b156-744944b1167d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.193454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5636dde-4e73-4f57-b2bc-2386af7ea8b0 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.196854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.197477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.197781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.198055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.209010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:28.214600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:28.215529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:28.216361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 519/1047] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:28.221821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.231189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.231835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.232113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.232381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.233111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] RBAC: Adding query filter params (role.id=78d11a72db5a4392971caa71134bde31, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=4e926d4a438e42ecaeab44f711e0b3c9) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:28.233416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.234873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a6a5ab-a66b-49c0-b71e-770711f317ff None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.240298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 529/1048] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/role_assignments?user.id=4e926d4a438e42ecaeab44f711e0b3c9&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=78d11a72db5a4392971caa71134bde31 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:28.247749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93007d-eec1-4f38-9ca1-fb0105850300 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.248031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93007d-eec1-4f38-9ca1-fb0105850300 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.248304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e93007d-eec1-4f38-9ca1-fb0105850300 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.249783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 530/1049] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:28.255558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.255861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.256109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.280701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:28.281972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:28.341498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.352643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e16cd9-d837-4da4-866a-4c65858f4b31 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.362317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 531/1050] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:28 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:28.394421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.395036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.395318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.395590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.396259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:28.396559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.397963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.400249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.412279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 520/1051] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:28.414411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.518998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fb51f-cc68-493f-8082-65f3e07e85fe None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.553886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.554538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.554823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.555102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] PATH_INFO: `/v3/users/f0d14198cbc141278e4dd1e08e1e0f38` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.568640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] RBAC: Authorizing `identity:get_user(user_id=f0d14198cbc141278e4dd1e08e1e0f38)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.570222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.576621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 532/1052] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/users/f0d14198cbc141278e4dd1e08e1e0f38 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:28.578166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.635621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.684896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.693376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd008f-88ab-47e6-9b96-8d4be2553ef0 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.732895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.733503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.734082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.734082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.746239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7053e7-8bc8-4c3c-a09d-36d601c34a11 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.781945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.782500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.782782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.783049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] PATH_INFO: `/v3/services/image` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.783640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.785001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:28.795258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:28.796297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 05:34:28.797156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:28.801363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 533/1053] 10.222.0.73 () {56 vars in 1105 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:28.802203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.809496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.815366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.816959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.911840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfa4ed-d1d7-4cd1-8cb3-c433945d66d2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:28.920320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:28.923389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 521/1054] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:28.929270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:28.953145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:28.953950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:28.954221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:28.954806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:28.955445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] RBAC: Adding query filter params (name=image) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:28.955754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:28.957393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:28.966733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 534/1055] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:28.967794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:29.035671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:29.042491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45313917-0243-45c4-8307-af6c0e3eab13 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:29.080708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:29.081337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:29.081649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:29.081928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:29.082487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e4cd4-e6d6-4db8-b41c-02f1efdf0b08 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:29.083627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] RBAC: Authorizing `identity:get_project(project_id=4fa60561153c4fae85803d93466f6a03)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:29.085227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:29.087855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 522/1056] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/projects/4fa60561153c4fae85803d93466f6a03 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:29.094118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:29.121188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:29.121675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:29.121959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:29.122225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:29.122859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] RBAC: Adding query filter params (type=image) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:29.123164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:29.124672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:29.134172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 535/1057] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:28 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:29.135113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:29.170780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:29.207414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9596bf51-43cb-4e3d-91af-61a82f7f4cdf None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:29.241638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d8c83-ae99-43ca-93d8-1c0580110acd None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:29.243712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:29.244323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:29.244603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:29.244872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:29.245627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=4fa60561153c4fae85803d93466f6a03, user.id=f0d14198cbc141278e4dd1e08e1e0f38) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:29.245939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:29.247375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9435d080-f97a-4f30-8e47-73a378362383 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:29.261595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 523/1058] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:29 2022] GET /identity/v3/role_assignments?user.id=f0d14198cbc141278e4dd1e08e1e0f38&scope.project.id=4fa60561153c4fae85803d93466f6a03&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:29.291182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3755a-ac2c-4d9e-a275-4ea9c3e5e766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:29.291826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3755a-ac2c-4d9e-a275-4ea9c3e5e766 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:29.292199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3755a-ac2c-4d9e-a275-4ea9c3e5e766 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:29.292544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3755a-ac2c-4d9e-a275-4ea9c3e5e766 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:29.293302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3755a-ac2c-4d9e-a275-4ea9c3e5e766 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:29.294851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3755a-ac2c-4d9e-a275-4ea9c3e5e766 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:29.322736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 536/1059] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:34:29 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:30.145408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37cff-d2e9-4c23-af18-21a103246587 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:30.145865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37cff-d2e9-4c23-af18-21a103246587 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:30.146049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f37cff-d2e9-4c23-af18-21a103246587 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:30.147486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 524/1060] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:30.152409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:30.152858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:30.152945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:30.171328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:30.172510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:30.249451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 537/1061] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:30.526910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37cff-d2e9-4c23-af18-21a103246587 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:30.562147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:30.643726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f37cff-d2e9-4c23-af18-21a103246587 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:30.681237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:30.682279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:30.682776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:30.683257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:30.694089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:30.699289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:30.700202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:30.701027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:30.703680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 525/1062] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 05:34:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:30.709418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:30.738435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:30.831611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c30d770-ed8d-4b2b-9138-592a822125f2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:30.871774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:30.872397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:30.872678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:30.872952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:30.873649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:30.873957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:30.875323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:30.884188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 538/1063] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:34:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:30.889845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:30.986602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:30.997826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6c65a-44b0-4e36-aef1-f22fa80d13f5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.034681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.035680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.036167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.036661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.048545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.050420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-75b90feb-2c08-4570-949d-377e36f1f8b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:31.066371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:31.067367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:31.068243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:31.068243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:31.068243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:31.070943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 526/1064] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:31.076767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.158007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:31.181488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d2085-b2ee-4a5d-ae18-38d558eacc28 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.202404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb29ac-b893-4526-af57-81b99d79e38a None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.202728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb29ac-b893-4526-af57-81b99d79e38a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.202966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fb29ac-b893-4526-af57-81b99d79e38a None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.204413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 527/1065] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:31.209066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574afcec-73b5-4a97-8d23-33963532024b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.209353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574afcec-73b5-4a97-8d23-33963532024b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.209665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574afcec-73b5-4a97-8d23-33963532024b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.218874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.219489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.219766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.220036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.220683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:31.220988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.222960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:31.227870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-574afcec-73b5-4a97-8d23-33963532024b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:31.229332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-574afcec-73b5-4a97-8d23-33963532024b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:31.240439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 539/1066] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:31.241715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.308264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 528/1067] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:31.313794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73549-4390-4b19-84dc-a501ae5d6fe5 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.313794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73549-4390-4b19-84dc-a501ae5d6fe5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.313794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe73549-4390-4b19-84dc-a501ae5d6fe5 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.313794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 529/1068] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:31.316491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.316777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.317060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.334760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:31.336020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:31.347896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f911cbe-aba4-4e55-ab3b-f79ca65980c3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.382787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.383363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.383639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.383910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.391212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.392646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:31.394415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:31.405010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:31.405974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:31.407021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 540/1069] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:31.410666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 530/1070] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:31.415260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.516760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404aa932-eafc-4042-b6f5-9ee61a7926a4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.551030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.551632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.551905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.552170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.552889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:31.553189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.554736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:31.572705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 541/1071] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:31.578142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.579668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.580827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:31.641084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:31.686314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4109fe-15f3-42ec-8d36-c5cd5b868bf5 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.686852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd291138-e852-4c68-ad19-a82e668a028a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.720544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.721175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.721411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.721520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.721747 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.722197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.722478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.722752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.732288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:31.733257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.734724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:31.737012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:31.737991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:31.738836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 542/1072] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:31.739505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:31.751739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:31.752800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:31.753736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:31.753736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:31.753736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:31.753736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 531/1073] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:31.753736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.814532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:31.840308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7760ca0-c070-427a-81f7-a65c1c5d13a2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.857168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:31.858589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da64ff7-cdf3-40eb-a85b-4e1cfc61483b None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:31.876472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.877078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.877359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.877673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.892130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:31.892731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:31.893011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:31.893283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:31.893975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:31.894278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.895650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:31.904178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 532/1074] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:31.905076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:31.948660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:31.950074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.006719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea5eb2-0258-4c03-b72b-1fa3a90794fd None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.022587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:32.040557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.041166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.041441 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.041744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.042376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:32.042679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.044043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.047920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 543/1075] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:32.048755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.063822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 533/1076] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:32.063822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.066578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:32.147326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047dea3-614b-47c2-9581-a454e9d7755d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.160321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3324ed-c1d9-4869-904f-7480e0ccf23e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.182749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.183381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.183662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.183937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] PATH_INFO: `/v3/users/f0d14198cbc141278e4dd1e08e1e0f38` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.185065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] RBAC: Authorizing `identity:get_user(user_id=f0d14198cbc141278e4dd1e08e1e0f38)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.186524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.188261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 544/1077] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:32 2022] GET /identity/v3/users/f0d14198cbc141278e4dd1e08e1e0f38 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:32.188948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.198452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.199140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.199371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.199644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.210170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.212136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:32.226085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:32.227874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:32.229022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:32.229022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:32.229022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:32.229022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 534/1078] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:32.229022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.261469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:32.278459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:32.295063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25775add-7249-4a27-b280-4fe5af11d680 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.328656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.329293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.329587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.329865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.336628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.337341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b5e089-a68f-47ad-8057-f635d4be00a4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.338089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:32.349237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:32.350327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 545/1079] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:32.351246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.375921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.376541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.377093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.377093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.378181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] RBAC: Authorizing `identity:get_project(project_id=4fa60561153c4fae85803d93466f6a03)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.379594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.386809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 535/1080] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:32 2022] GET /identity/v3/projects/4fa60561153c4fae85803d93466f6a03 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:32.386809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.450764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7200b-ff0d-4d14-b28e-59ee62f3de6d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.479781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:32.483469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.484075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.484354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.484623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.485256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:32.485570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.486167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54583af0-d22f-4bb9-9f8a-b1c410e8d419 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.486934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.498968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 546/1081] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:32.499749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.503546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:32.528911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.530246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.530741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.531021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.531743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:32.532055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.533537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.548016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 536/1082] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:32.554186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:32.633711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f9e9c-2259-448b-990d-9240f9e089cf None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.672469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdc72a7-924c-465e-b48c-831ef03beb0a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:32.675751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c0555-db10-491e-9710-218e5fa425ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.676426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c0555-db10-491e-9710-218e5fa425ff None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.676707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c0555-db10-491e-9710-218e5fa425ff None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.676707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c0555-db10-491e-9710-218e5fa425ff None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03/users/f0d14198cbc141278e4dd1e08e1e0f38/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.680653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c0555-db10-491e-9710-218e5fa425ff None admin] RBAC: Authorizing `identity:create_grant(project_id=4fa60561153c4fae85803d93466f6a03, user_id=f0d14198cbc141278e4dd1e08e1e0f38, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.682189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c0555-db10-491e-9710-218e5fa425ff None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.699945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 547/1083] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:34:32 2022] PUT /identity/v3/projects/4fa60561153c4fae85803d93466f6a03/users/f0d14198cbc141278e4dd1e08e1e0f38/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:34:32.720776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9a253-1e5b-4188-a0a3-1f84a5422235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:32.721378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a253-1e5b-4188-a0a3-1f84a5422235 None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:32.721685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a253-1e5b-4188-a0a3-1f84a5422235 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:32.722568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a253-1e5b-4188-a0a3-1f84a5422235 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/4e926d4a438e42ecaeab44f711e0b3c9/roles/78d11a72db5a4392971caa71134bde31` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:32.725259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9a253-1e5b-4188-a0a3-1f84a5422235 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=4e926d4a438e42ecaeab44f711e0b3c9, role_id=78d11a72db5a4392971caa71134bde31)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:32.727334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c9a253-1e5b-4188-a0a3-1f84a5422235 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:32.744891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 537/1084] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:34:32 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/4e926d4a438e42ecaeab44f711e0b3c9/roles/78d11a72db5a4392971caa71134bde31 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:34:34.600420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db4df3-77a5-4a6e-95be-06745a7e14f8 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:34.600940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db4df3-77a5-4a6e-95be-06745a7e14f8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:34.600940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0db4df3-77a5-4a6e-95be-06745a7e14f8 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:34.602506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 548/1085] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:34.608066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:34.608500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:34.608618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:34.626946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:34.628117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:34.639310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:34.649369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27d6ce-7988-4bca-bd95-60819e2ff8ac None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:34.649687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27d6ce-7988-4bca-bd95-60819e2ff8ac None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:34.649968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27d6ce-7988-4bca-bd95-60819e2ff8ac None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:34.651447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 549/1086] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:34:34.656235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:34.656519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:34.656857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:34.674679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:34.675973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:34.696362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 538/1087] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:34.735406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:34.744800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 550/1088] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:34.964393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.011896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.064530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80c0a8-37a2-4a0e-aa80-bfb981c097c0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.099062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.099692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.099973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.100250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.111551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:35.112024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8529e7-de8f-4395-9f38-73ebd5f4368a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:35.116302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:35.117874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:35.119076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 539/1089] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:34:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:35.123970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.132762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.146193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.146823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.147105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.147377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.157658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:35.162446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:35.163391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:35.164787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 551/1090] 10.222.0.73 () {56 vars in 1120 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:35.170095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.225170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9d545-c5aa-4be9-9cd6-9ce49a574d20 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.225371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.260215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.260827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.261100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.261368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.262081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:35.262388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.263750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.274550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c0b5c4-a298-4425-b8be-475052508204 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.275311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 540/1091] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:35.280968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.308637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.309242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.309518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.309817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.310464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:35.310762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.312122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.321596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 552/1092] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:35.325499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.348377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.383873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620677fc-05b3-4bf7-9000-b07b34cc78f4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.417519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.418251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.418530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.418801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] PATH_INFO: `/v3/users/f0d14198cbc141278e4dd1e08e1e0f38` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.419931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] RBAC: Authorizing `identity:get_user(user_id=f0d14198cbc141278e4dd1e08e1e0f38)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.421361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.422734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502b2f4-0899-49a0-a079-8a9e78ad9127 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.423225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 541/1093] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/users/f0d14198cbc141278e4dd1e08e1e0f38 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:35.428717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.438579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.456590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.457216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.457492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.457789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.468523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.469985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:35.484532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:35.485482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:35.486406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:35.486406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:35.486406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:35.486406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 553/1094] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:35.491289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.534083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500fe89a-2b55-41a6-9102-9773f10e2e3c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.569527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.570673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.570947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.571395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.574914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.594250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac606c67-3ca9-459e-bc83-0b0765d5e5ad None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.626765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.627393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.627625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.627893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.628518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:35.628825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.630203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.639817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.641262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.642101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 554/1095] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:35.647641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.674560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.740815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 542/1096] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:34:35.747358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.747716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9668e7-dbde-4311-a217-6078d9aa507c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.781978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.782608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.782890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.783164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.783263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.790816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.792247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:34:35.805047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:35.806004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:35.806935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 555/1097] 10.222.0.73 () {56 vars in 1093 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:34:35.811826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.849509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69647f36-8e33-43dc-b923-d9c4da81915f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.883982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02028e07-2865-4459-9a32-214bed686963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.884606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02028e07-2865-4459-9a32-214bed686963 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.884884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02028e07-2865-4459-9a32-214bed686963 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.885157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02028e07-2865-4459-9a32-214bed686963 None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.886340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02028e07-2865-4459-9a32-214bed686963 None admin] RBAC: Authorizing `identity:get_project(project_id=4fa60561153c4fae85803d93466f6a03)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.887779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02028e07-2865-4459-9a32-214bed686963 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.889416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 543/1098] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/projects/4fa60561153c4fae85803d93466f6a03 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:35.895452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02028e07-2865-4459-9a32-214bed686963 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.901869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:35.914819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f50d2a-8a6a-46aa-8d9e-845486bbb769 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:35.949457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:35.950091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:35.950367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:35.950635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:35.951356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:35.951658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:35.953065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:35.970762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 556/1099] 10.222.0.73 () {56 vars in 1133 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:35.976618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:35.999422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02028e07-2865-4459-9a32-214bed686963 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:36.001765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:36.036285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:36.036892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:36.037172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:36.037446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:36.038211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=4fa60561153c4fae85803d93466f6a03, user.id=f0d14198cbc141278e4dd1e08e1e0f38) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:36.038523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:36.039974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:36.045647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 544/1100] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/role_assignments?user.id=f0d14198cbc141278e4dd1e08e1e0f38&scope.project.id=4fa60561153c4fae85803d93466f6a03&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:36.081025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b37a8-d4cd-4291-a5e3-791dff4d0322 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:36.118275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:36.118901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:36.119182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:36.119457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:36.144252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:36.209836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:36.211297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:36.325846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 557/1101] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:34:36.331646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:36.437404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2fab38-f055-471e-970d-e32aa8599232 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:36.471540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:36.472172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:36.472451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:36.472720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:36.485635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:36.487094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:34:36.502400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:36.503575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:34:36.504582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:34:36.504582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:34:36.504582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:36.504582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 545/1102] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:36.509350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:36.569769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:36.628738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d1908-4818-4d66-b30b-9865dd7d089f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:36.665997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:36.666612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:36.666887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:36.667287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:36.667930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:36.668230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:36.669612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:36.688287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 558/1103] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:34:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:36.689811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:36.712969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:36.791404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27851b47-f2ea-4ff5-b862-7204f41e31a2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:36.823502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:36.824063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:36.824341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:36.824611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:36.831105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:36.832543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:34:36.844017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:36.845063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:34:36.845983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 546/1104] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:34:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:36.850919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:36.953003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efcf3eb-5c3b-4cc6-a70a-0f990862cd31 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:36.960346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:36.990180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:36.990794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:36.991072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:36.991343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:36.992064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:36.992372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:36.994162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e1db85-8c8c-4ea1-b2d4-9e42757a879e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:37.010519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 559/1105] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:34:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:37.011608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:37.095670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:37.114823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f6bf5-8473-4ec5-8950-021d38a290d1 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:37.147026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:37.147621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:37.147898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:37.148767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:37.149487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] RBAC: Adding query filter params (role.id=78d11a72db5a4392971caa71134bde31, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=4e926d4a438e42ecaeab44f711e0b3c9) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:37.149823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:37.151250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67338b2a-eb60-423d-8db9-97529688dac5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:37.156992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 547/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:37 2022] GET /identity/v3/role_assignments?user.id=4e926d4a438e42ecaeab44f711e0b3c9&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=78d11a72db5a4392971caa71134bde31 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:34:38.000511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deccc8-6e47-49cf-ab98-20a3c468303f None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:38.001025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deccc8-6e47-49cf-ab98-20a3c468303f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:38.001025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6deccc8-6e47-49cf-ab98-20a3c468303f None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:38.002392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 560/1107] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:38.007752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:38.008134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:38.008288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:38.025917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:38.027087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:38.090853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 548/1108] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:38.358766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deccc8-6e47-49cf-ab98-20a3c468303f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:38.456910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6deccc8-6e47-49cf-ab98-20a3c468303f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:38.489833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ed7b3-ffc8-4dc5-bccd-f20c1e4831e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:38.490442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed7b3-ffc8-4dc5-bccd-f20c1e4831e1 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:38.490714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed7b3-ffc8-4dc5-bccd-f20c1e4831e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:38.490978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ed7b3-ffc8-4dc5-bccd-f20c1e4831e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:38.492082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed7b3-ffc8-4dc5-bccd-f20c1e4831e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:38.493487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ed7b3-ffc8-4dc5-bccd-f20c1e4831e1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:38.495040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 561/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:38.500665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:38.573613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:38.598152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b59fc-5a73-47cf-ad6d-46cd249e35e1 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:38.630192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203d8f-32e7-4c6f-b8d2-f28906833798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:38.630784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203d8f-32e7-4c6f-b8d2-f28906833798 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:38.631058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203d8f-32e7-4c6f-b8d2-f28906833798 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:38.631325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203d8f-32e7-4c6f-b8d2-f28906833798 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:38.631967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203d8f-32e7-4c6f-b8d2-f28906833798 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:38.633938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203d8f-32e7-4c6f-b8d2-f28906833798 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:38.655568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 549/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:34:38 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:34:40.557929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdd137-4a7a-41e2-a909-afc7ec3a16c5 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:40.558489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdd137-4a7a-41e2-a909-afc7ec3a16c5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:40.558489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bdd137-4a7a-41e2-a909-afc7ec3a16c5 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:40.559739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 562/1111] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:40.564993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:40.565412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:40.565539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:40.583844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:40.585021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:40.648798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 550/1112] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:40.917274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdd137-4a7a-41e2-a909-afc7ec3a16c5 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:40.951830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:41.017471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bdd137-4a7a-41e2-a909-afc7ec3a16c5 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:41.053097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:41.053740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:41.054016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:41.054873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] PATH_INFO: `/v3/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:41.056739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] RBAC: Authorizing `identity:get_role(role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:41.058165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:41.059577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 563/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:40 2022] GET /identity/v3/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:41.067273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:41.125200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:41.165626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98453e0c-6f23-4e8d-80f9-9bbba38782cd None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:41.198279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:41.198835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:41.199115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:41.199384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] PATH_INFO: `/v3/users/ae8cc58efadd4cf78418c1d23d21e752` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:41.212459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] RBAC: Authorizing `identity:get_user(user_id=ae8cc58efadd4cf78418c1d23d21e752)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:41.213920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:41.215482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 551/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:41 2022] GET /identity/v3/users/ae8cc58efadd4cf78418c1d23d21e752 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:41.221180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:41.319405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87b0a-f930-4475-b54f-a0bae9d170ca None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:41.320951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:41.354223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:41.354819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:41.355102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:41.355399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:41.420241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:41.421702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:41.517104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 564/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:34:41.524130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:41.640695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c4ead-e01a-4ace-9a36-1c7378207795 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:41.662631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:41.681028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46e28-b2fb-45e6-99d2-d765e7532d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:41.681677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46e28-b2fb-45e6-99d2-d765e7532d90 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:41.681953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46e28-b2fb-45e6-99d2-d765e7532d90 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:41.682222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46e28-b2fb-45e6-99d2-d765e7532d90 None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:41.683339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46e28-b2fb-45e6-99d2-d765e7532d90 None admin] RBAC: Authorizing `identity:get_project(project_id=4fa60561153c4fae85803d93466f6a03)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:41.684759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46e28-b2fb-45e6-99d2-d765e7532d90 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:41.686410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 552/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:41 2022] GET /identity/v3/projects/4fa60561153c4fae85803d93466f6a03 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:41.692959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:41.695165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:41.791041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742e4c5-9a05-42de-9b42-caf52bcc25ba None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:41.823429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:41.824010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:41.824287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:41.824557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:41.825266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] RBAC: Adding query filter params (role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=4fa60561153c4fae85803d93466f6a03, user.id=ae8cc58efadd4cf78418c1d23d21e752) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:41.825586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:41.827020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73afa943-dccd-44a6-a7f5-2c9cf9cc5c6b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:41.832179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 565/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:41 2022] GET /identity/v3/role_assignments?user.id=ae8cc58efadd4cf78418c1d23d21e752&scope.project.id=4fa60561153c4fae85803d93466f6a03&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:43.739241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b0e84-8f29-4fe0-aed0-7175d36cf40b None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:43.739816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b0e84-8f29-4fe0-aed0-7175d36cf40b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:43.739816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b0e84-8f29-4fe0-aed0-7175d36cf40b None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:43.741120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 553/1118] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:43.746738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:43.747167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:43.747299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:43.766183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:43.767373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:43.825348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:43.833050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 566/1119] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:44.100878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b0e84-8f29-4fe0-aed0-7175d36cf40b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:44.200362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b0e84-8f29-4fe0-aed0-7175d36cf40b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:44.234050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:44.234674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:44.234955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:44.235226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] PATH_INFO: `/v3/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:44.236624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] RBAC: Authorizing `identity:get_role(role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:44.238021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:44.239312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 554/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:44 2022] GET /identity/v3/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:44.245095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:44.340121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8bd2f-6609-403b-b089-08679f432c55 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:44.373047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:44.373633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:44.373915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:44.374185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] PATH_INFO: `/v3/users/ae8cc58efadd4cf78418c1d23d21e752` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:44.375285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] RBAC: Authorizing `identity:get_user(user_id=ae8cc58efadd4cf78418c1d23d21e752)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:44.376707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:44.378316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 567/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:44 2022] GET /identity/v3/users/ae8cc58efadd4cf78418c1d23d21e752 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:44.384019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:44.438083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:44.481838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3eeb0-bd70-445e-b196-b12650ae6fae None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:44.513745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eff9c-1e4b-48d8-9679-b613913b3b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:44.514357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eff9c-1e4b-48d8-9679-b613913b3b02 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:44.514631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eff9c-1e4b-48d8-9679-b613913b3b02 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:44.514899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eff9c-1e4b-48d8-9679-b613913b3b02 None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:44.515992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eff9c-1e4b-48d8-9679-b613913b3b02 None admin] RBAC: Authorizing `identity:get_project(project_id=4fa60561153c4fae85803d93466f6a03)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:44.517411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eff9c-1e4b-48d8-9679-b613913b3b02 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:44.519026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 555/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:44 2022] GET /identity/v3/projects/4fa60561153c4fae85803d93466f6a03 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:44.525517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:44.586193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:44.623468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bb01d-f784-4a0c-9237-dd0ee7be665c None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:44.655901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e56aa2e-88b4-42a5-bb62-c14d2a6f8ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:44.656546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56aa2e-88b4-42a5-bb62-c14d2a6f8ea1 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:44.656824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56aa2e-88b4-42a5-bb62-c14d2a6f8ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:44.657096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e56aa2e-88b4-42a5-bb62-c14d2a6f8ea1 None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03/users/ae8cc58efadd4cf78418c1d23d21e752/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:44.659211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56aa2e-88b4-42a5-bb62-c14d2a6f8ea1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4fa60561153c4fae85803d93466f6a03, user_id=ae8cc58efadd4cf78418c1d23d21e752, role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:44.660629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e56aa2e-88b4-42a5-bb62-c14d2a6f8ea1 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:44.672929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 568/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:34:44 2022] PUT /identity/v3/projects/4fa60561153c4fae85803d93466f6a03/users/ae8cc58efadd4cf78418c1d23d21e752/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:34:46.540341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e93999-f330-481c-86eb-3aed3765b3bf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:46.540903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e93999-f330-481c-86eb-3aed3765b3bf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:46.540903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e93999-f330-481c-86eb-3aed3765b3bf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:46.542197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 556/1124] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:46.547447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:46.547887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:46.547978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:46.566333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:46.567498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:46.630876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 569/1125] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:34:46.899661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e93999-f330-481c-86eb-3aed3765b3bf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:46.999778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e93999-f330-481c-86eb-3aed3765b3bf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:47.021408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:47.033308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:47.033961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:47.034244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:47.034519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] PATH_INFO: `/v3/roles/def9344cb5ba45449e0f5e98c5fe5222` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:47.035894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] RBAC: Authorizing `identity:get_role(role_id=def9344cb5ba45449e0f5e98c5fe5222)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:47.037263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:47.038600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 557/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:46 2022] GET /identity/v3/roles/def9344cb5ba45449e0f5e98c5fe5222 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:47.044811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:47.099216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:47.143589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11945f92-e3b7-4616-bdbe-5f42c64a5c7a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:47.181442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:47.182042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:47.182321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:47.182589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] PATH_INFO: `/v3/users/ae8cc58efadd4cf78418c1d23d21e752` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:47.183707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] RBAC: Authorizing `identity:get_user(user_id=ae8cc58efadd4cf78418c1d23d21e752)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:47.185132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:47.186746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 570/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:47 2022] GET /identity/v3/users/ae8cc58efadd4cf78418c1d23d21e752 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:47.192646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:47.290118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f0551-a8f0-4619-b901-e2c979259d62 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:47.322996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:47.323587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:47.323864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:47.324132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:47.379209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:47.390292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:47.391735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:47.487356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 558/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:34:47.493322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:47.589936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461faef-21d3-4fa0-86c4-0dd15131ba09 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:47.611442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:47.623157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4aa02e-42b6-4d30-81c8-e657161f8689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:47.623780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aa02e-42b6-4d30-81c8-e657161f8689 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:47.624059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aa02e-42b6-4d30-81c8-e657161f8689 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:47.624329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4aa02e-42b6-4d30-81c8-e657161f8689 None admin] PATH_INFO: `/v3/projects/4fa60561153c4fae85803d93466f6a03` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:47.625431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aa02e-42b6-4d30-81c8-e657161f8689 None admin] RBAC: Authorizing `identity:get_project(project_id=4fa60561153c4fae85803d93466f6a03)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:47.626890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4aa02e-42b6-4d30-81c8-e657161f8689 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:47.628569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 571/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:47 2022] GET /identity/v3/projects/4fa60561153c4fae85803d93466f6a03 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:47.635364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:47.722595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:47.732512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3358c52-c68b-4123-a3dd-2c4cb3ab3c60 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:47.765407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:47.765987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:47.766271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:47.766546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:47.767266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] RBAC: Adding query filter params (role.id=def9344cb5ba45449e0f5e98c5fe5222, scope.project.id=4fa60561153c4fae85803d93466f6a03, user.id=ae8cc58efadd4cf78418c1d23d21e752) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:47.767575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:47.769000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c92b1ab-556d-47b9-adcd-de3b33211da0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:47.775158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 559/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:47 2022] GET /identity/v3/role_assignments?user.id=ae8cc58efadd4cf78418c1d23d21e752&scope.project.id=4fa60561153c4fae85803d93466f6a03&role.id=def9344cb5ba45449e0f5e98c5fe5222 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:49.670476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8c912-7903-4426-a8b3-762b7eb6f3fe None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:49.671022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8c912-7903-4426-a8b3-762b7eb6f3fe None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:49.671130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c8c912-7903-4426-a8b3-762b7eb6f3fe None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:49.672384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 572/1131] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:49.677489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:49.677898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:49.678075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:49.697211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:49.698443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:49.763927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 560/1132] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:50.030196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8c912-7903-4426-a8b3-762b7eb6f3fe None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:50.129836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c8c912-7903-4426-a8b3-762b7eb6f3fe None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:50.162060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c2c48-81b9-4d38-a4cb-ed8546087377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:50.162672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c2c48-81b9-4d38-a4cb-ed8546087377 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:50.162949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c2c48-81b9-4d38-a4cb-ed8546087377 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:50.163218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c2c48-81b9-4d38-a4cb-ed8546087377 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:50.164915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c2c48-81b9-4d38-a4cb-ed8546087377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:50.166377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c2c48-81b9-4d38-a4cb-ed8546087377 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:50.167903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 573/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:50.173672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:50.254634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:50.271046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b2c6d-4af3-4794-80ed-3e1ee2d2337b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:50.305644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545b9e62-d4a1-4152-acfb-1be6ab6fcf46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:50.306201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b9e62-d4a1-4152-acfb-1be6ab6fcf46 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:50.306483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b9e62-d4a1-4152-acfb-1be6ab6fcf46 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:50.306754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545b9e62-d4a1-4152-acfb-1be6ab6fcf46 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:50.307382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b9e62-d4a1-4152-acfb-1be6ab6fcf46 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:50.308808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545b9e62-d4a1-4152-acfb-1be6ab6fcf46 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:50.333637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 561/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:34:50 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:52.231427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a4b92-d5e8-4fad-9b97-79c1e8952070 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:52.231987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a4b92-d5e8-4fad-9b97-79c1e8952070 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:52.231987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1a4b92-d5e8-4fad-9b97-79c1e8952070 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:52.233325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 574/1135] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:52.238458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:52.238875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:52.239008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:52.257490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:52.258738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:52.324203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 562/1136] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:52.590031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a4b92-d5e8-4fad-9b97-79c1e8952070 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:52.647125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:52.688632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1a4b92-d5e8-4fad-9b97-79c1e8952070 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:52.721499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14816fa9-f512-4e1f-86b5-dc41866af2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:52.722142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14816fa9-f512-4e1f-86b5-dc41866af2cb None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:52.722422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14816fa9-f512-4e1f-86b5-dc41866af2cb None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:52.722692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14816fa9-f512-4e1f-86b5-dc41866af2cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:52.723786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14816fa9-f512-4e1f-86b5-dc41866af2cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:52.725208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14816fa9-f512-4e1f-86b5-dc41866af2cb None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:52.727512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 575/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:34:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:52.736028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:52.812144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:52.846661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b134e6b-861a-4445-8b72-c66464cd2d07 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:52.879050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc928ddc-e30e-41d5-a64d-c514221ce270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:52.879646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928ddc-e30e-41d5-a64d-c514221ce270 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:52.879928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928ddc-e30e-41d5-a64d-c514221ce270 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:52.880199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc928ddc-e30e-41d5-a64d-c514221ce270 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:52.880829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc928ddc-e30e-41d5-a64d-c514221ce270 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:52.882298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc928ddc-e30e-41d5-a64d-c514221ce270 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:52.904108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 563/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:34:52 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:34:54.843041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248548d-8c6c-4e8a-9f8a-675c2be71b59 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:54.843569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248548d-8c6c-4e8a-9f8a-675c2be71b59 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:54.843569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7248548d-8c6c-4e8a-9f8a-675c2be71b59 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:54.844919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 576/1139] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:54.850437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:54.850857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:54.850964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:54.869479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:54.870697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:54.935876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 564/1140] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:55.202883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248548d-8c6c-4e8a-9f8a-675c2be71b59 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:55.299816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7248548d-8c6c-4e8a-9f8a-675c2be71b59 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:55.324392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:55.332967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:55.333602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:55.333879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:55.334143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:55.343700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:55.348369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:55.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:55.354879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 577/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:34:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:55.356336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:55.406880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:55.452843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f926407-9f25-43d5-bd3c-05165b47fc71 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:55.486325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:55.486859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:55.487134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:55.487401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:55.488051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:55.488348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:55.489736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:55.497640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 565/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:34:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:55.503576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:55.604088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f833f-814e-4064-afd1-46a143a0e852 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:55.635974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:55.636579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:55.636855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:55.637121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] PATH_INFO: `/v3/users/d5b95ab1e73e4b96bd3f751442dbad63` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:55.650099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] RBAC: Authorizing `identity:get_user(user_id=d5b95ab1e73e4b96bd3f751442dbad63)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:55.651527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:55.653270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 578/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:55 2022] GET /identity/v3/users/d5b95ab1e73e4b96bd3f751442dbad63 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:55.659233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:55.758018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ada289-eee9-46cc-ba2f-eeef937c770c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:55.771043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:55.792093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:55.792713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:55.792977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:55.793248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:55.859427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:55.860884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:55.957662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 566/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:34:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 05:34:55.964161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:56.008624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:56.062436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3911f6f-2672-4ca3-adb1-86a3aff9eeda None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:56.097200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4328edf8-7101-47b6-b840-4f7872ccc527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:56.097856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328edf8-7101-47b6-b840-4f7872ccc527 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:56.098135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328edf8-7101-47b6-b840-4f7872ccc527 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:56.098405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4328edf8-7101-47b6-b840-4f7872ccc527 None admin] PATH_INFO: `/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:56.099514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328edf8-7101-47b6-b840-4f7872ccc527 None admin] RBAC: Authorizing `identity:get_project(project_id=2482f78ed7e44c6ba6eda80dd06471c3)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:56.100935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4328edf8-7101-47b6-b840-4f7872ccc527 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:56.102654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 579/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:55 2022] GET /identity/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:56.109091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:56.123878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:56.207052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b404565-7101-44de-8e1d-b9b7d66253ae None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:56.240391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:56.240981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:56.241261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:56.241534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:56.242829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=2482f78ed7e44c6ba6eda80dd06471c3, user.id=d5b95ab1e73e4b96bd3f751442dbad63) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:56.243137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:56.244560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4423e3f-3490-4f31-91da-c9992eba21d9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:56.249866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 567/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:34:56 2022] GET /identity/v3/role_assignments?user.id=d5b95ab1e73e4b96bd3f751442dbad63&scope.project.id=2482f78ed7e44c6ba6eda80dd06471c3&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:58.113802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846568-4a19-4b5b-81f4-7497b3bc0928 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:58.114332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846568-4a19-4b5b-81f4-7497b3bc0928 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:58.114332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846568-4a19-4b5b-81f4-7497b3bc0928 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:58.115678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 580/1147] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:34:58.121391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fd068-90e8-439b-b84c-93a970612027 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:58.121980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fd068-90e8-439b-b84c-93a970612027 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:58.121980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fd068-90e8-439b-b84c-93a970612027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:58.139475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-214fd068-90e8-439b-b84c-93a970612027 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:34:58.140658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-214fd068-90e8-439b-b84c-93a970612027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:34:58.204487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 568/1148] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:34:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:34:58.476116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846568-4a19-4b5b-81f4-7497b3bc0928 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:58.575671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846568-4a19-4b5b-81f4-7497b3bc0928 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:58.588931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:58.610283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:58.610903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:58.611183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:58.611456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:58.621249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:34:58.625997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:34:58.627104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:34:58.628308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 581/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:34:58.633093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fd068-90e8-439b-b84c-93a970612027 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:58.643093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:58.731359 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fd068-90e8-439b-b84c-93a970612027 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:58.763615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:58.764211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:58.764488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:58.764757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:58.765410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:34:58.765737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:58.767108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:58.775339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 569/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:34:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:58.781196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:58.878267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c83493b-f8bb-4f68-b11e-82544f7a6a03 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:58.911682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:58.912300 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:58.912635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:58.912915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] PATH_INFO: `/v3/users/d5b95ab1e73e4b96bd3f751442dbad63` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:58.914091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] RBAC: Authorizing `identity:get_user(user_id=d5b95ab1e73e4b96bd3f751442dbad63)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:58.915509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:58.917073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 582/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:34:58 2022] GET /identity/v3/users/d5b95ab1e73e4b96bd3f751442dbad63 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:58.922855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:58.990287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:59.020649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a094df56-9dd4-49c6-928e-ff792abe1303 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:59.052626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc6700-e0c8-4b40-9eb0-10572800eef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:59.053241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc6700-e0c8-4b40-9eb0-10572800eef6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:59.053517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc6700-e0c8-4b40-9eb0-10572800eef6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:59.053814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc6700-e0c8-4b40-9eb0-10572800eef6 None admin] PATH_INFO: `/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:59.054963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc6700-e0c8-4b40-9eb0-10572800eef6 None admin] RBAC: Authorizing `identity:get_project(project_id=2482f78ed7e44c6ba6eda80dd06471c3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:59.056391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc6700-e0c8-4b40-9eb0-10572800eef6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:59.058032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 570/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:34:58 2022] GET /identity/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:34:59.064088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:34:59.106414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:34:59.163651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e2a7ef-310a-456d-8b35-bdaad074571b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:34:59.196561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c2b54a-2059-453d-9205-43a7148f572f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:34:59.197207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b54a-2059-453d-9205-43a7148f572f None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:34:59.197487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b54a-2059-453d-9205-43a7148f572f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:34:59.197790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c2b54a-2059-453d-9205-43a7148f572f None admin] PATH_INFO: `/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3/users/d5b95ab1e73e4b96bd3f751442dbad63/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:34:59.199669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2b54a-2059-453d-9205-43a7148f572f None admin] RBAC: Authorizing `identity:create_grant(project_id=2482f78ed7e44c6ba6eda80dd06471c3, user_id=d5b95ab1e73e4b96bd3f751442dbad63, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:34:59.201098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c2b54a-2059-453d-9205-43a7148f572f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:34:59.214639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 583/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:34:59 2022] PUT /identity/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3/users/d5b95ab1e73e4b96bd3f751442dbad63/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:35:01.082889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90febb-ba03-413d-897e-e4d3f7e1e042 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:01.083460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90febb-ba03-413d-897e-e4d3f7e1e042 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:01.083460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b90febb-ba03-413d-897e-e4d3f7e1e042 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:01.084804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 571/1154] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:01.092111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:01.092612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:01.092704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:01.112121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:01.113317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:01.178311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 584/1155] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:35:01.443152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90febb-ba03-413d-897e-e4d3f7e1e042 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:01.542759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b90febb-ba03-413d-897e-e4d3f7e1e042 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:01.576291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:01.576904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:01.577175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:01.577438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:01.583952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:01.588112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:35:01.593307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:35:01.594442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:35:01.595509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 572/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:35:01.600504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:01.629750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:01.697578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae67ba-3c66-41ff-9022-b8af7ceb6a28 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:01.729661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:01.730257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:01.730532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:01.730799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:01.731445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:01.731741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:01.733093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:01.740865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 585/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:35:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:01.747218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:01.842439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12e27c-7af7-4db2-879f-528d7194a1c2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:01.876832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:01.877430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:01.877732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:01.878005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] PATH_INFO: `/v3/users/d5b95ab1e73e4b96bd3f751442dbad63` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:01.879095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=d5b95ab1e73e4b96bd3f751442dbad63)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:01.880506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:01.882054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 573/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:35:01 2022] GET /identity/v3/users/d5b95ab1e73e4b96bd3f751442dbad63 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:01.887942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:01.976979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:01.986564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f2401-c907-437c-9fad-d6980a7349d2 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:02.019345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:02.019947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:02.020226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:02.020496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:02.087269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:02.088718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:02.175065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:02.184525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 586/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:35:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:35:02.190667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:02.254545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:02.289499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc825817-4922-4735-82da-dbb641a66cb2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:02.321667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669ad40-182c-4a84-96ee-eda3b6ba2884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:02.322276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669ad40-182c-4a84-96ee-eda3b6ba2884 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:02.322548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669ad40-182c-4a84-96ee-eda3b6ba2884 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:02.322812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669ad40-182c-4a84-96ee-eda3b6ba2884 None admin] PATH_INFO: `/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:02.323899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669ad40-182c-4a84-96ee-eda3b6ba2884 None admin] RBAC: Authorizing `identity:get_project(project_id=2482f78ed7e44c6ba6eda80dd06471c3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:02.325318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669ad40-182c-4a84-96ee-eda3b6ba2884 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:02.326939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 574/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:35:02 2022] GET /identity/v3/projects/2482f78ed7e44c6ba6eda80dd06471c3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:02.333070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:02.427991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d86664-5466-4853-babb-db58e79cd959 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:02.460074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:02.461196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:02.461479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:02.461856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:02.462542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=2482f78ed7e44c6ba6eda80dd06471c3, user.id=d5b95ab1e73e4b96bd3f751442dbad63) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:02.462857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:02.464278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32faf7c2-6b61-45bd-93b4-455a433bb33e None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:02.469942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 587/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:35:02 2022] GET /identity/v3/role_assignments?user.id=d5b95ab1e73e4b96bd3f751442dbad63&scope.project.id=2482f78ed7e44c6ba6eda80dd06471c3&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:04.380305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df89107-7410-4f45-abf3-003f976b2073 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:04.380831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df89107-7410-4f45-abf3-003f976b2073 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:04.380831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df89107-7410-4f45-abf3-003f976b2073 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:04.382206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 575/1162] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:04.387370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:04.387775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:04.387916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:04.406117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:04.407322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:04.456443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:04.473224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 588/1163] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:35:04.736109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df89107-7410-4f45-abf3-003f976b2073 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:04.835094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df89107-7410-4f45-abf3-003f976b2073 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:04.869111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:04.869725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:04.870005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:04.870278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:04.872805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:04.937373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:04.938861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:05.033435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 576/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:35:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:35:05.039296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:05.135416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091daa3-6d46-411d-bb9a-2fa3393079c9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:05.169250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:05.169906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:05.170187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:05.170459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:05.176942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:05.178416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-37a3e868-1dbe-43b9-8a7e-57beef50b8d0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:35:05.189490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:35:05.190691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:35:05.191995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 589/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:35:05 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:35:05.196699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:05.230491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:05.296021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672e676-2984-4b4f-84af-a569f092793c None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:05.328226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:05.328818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:05.329097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:05.329364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:05.330024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:05.330325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:05.331678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec717da7-9ad4-43ba-966e-d11cc5852464 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:05.339060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 577/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Wed Aug 24 05:35:05 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:35:07.201372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92572e-9483-40c8-838c-725374de45a5 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:07.201933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92572e-9483-40c8-838c-725374de45a5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:07.201933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92572e-9483-40c8-838c-725374de45a5 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:07.203284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 590/1167] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:07.208467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d7395-0aab-4b31-a1cb-49369f08ec6e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:07.208851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d7395-0aab-4b31-a1cb-49369f08ec6e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:07.209106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d7395-0aab-4b31-a1cb-49369f08ec6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:07.227381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-534d7395-0aab-4b31-a1cb-49369f08ec6e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:07.228811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-534d7395-0aab-4b31-a1cb-49369f08ec6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:07.298527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 578/1168] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:35:07.563126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92572e-9483-40c8-838c-725374de45a5 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:07.610737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:07.663458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92572e-9483-40c8-838c-725374de45a5 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:07.696362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13435e9d-77ef-4872-ba43-93bdc61ce275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:07.696954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13435e9d-77ef-4872-ba43-93bdc61ce275 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:07.697231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13435e9d-77ef-4872-ba43-93bdc61ce275 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:07.697500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13435e9d-77ef-4872-ba43-93bdc61ce275 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:07.698229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13435e9d-77ef-4872-ba43-93bdc61ce275 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:07.699581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13435e9d-77ef-4872-ba43-93bdc61ce275 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:07.714847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 591/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Wed Aug 24 05:35:07 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:35:09.663263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37b5a7-cbe0-470a-a3b4-e0e85af25948 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:09.663928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37b5a7-cbe0-470a-a3b4-e0e85af25948 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:09.663928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37b5a7-cbe0-470a-a3b4-e0e85af25948 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:09.665193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 579/1170] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:35:09.671079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:09.671608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:09.671608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:09.690180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:09.691388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:09.763098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 592/1171] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:35:10.034583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37b5a7-cbe0-470a-a3b4-e0e85af25948 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:10.063123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:10.137176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37b5a7-cbe0-470a-a3b4-e0e85af25948 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:10.170592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7c814d-c86c-45c1-8fea-d1fd11517cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:10.171217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c814d-c86c-45c1-8fea-d1fd11517cbd None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:10.171494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c814d-c86c-45c1-8fea-d1fd11517cbd None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:10.171763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7c814d-c86c-45c1-8fea-d1fd11517cbd None admin] PATH_INFO: `/v3/domains/6e12820be45f46769100066a9d97aaa0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:10.182609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c814d-c86c-45c1-8fea-d1fd11517cbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=6e12820be45f46769100066a9d97aaa0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:10.184164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7c814d-c86c-45c1-8fea-d1fd11517cbd None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:10.185805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 580/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Wed Aug 24 05:35:10 2022] GET /identity/v3/domains/6e12820be45f46769100066a9d97aaa0 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:10.191651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:10.224579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:10.288280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30983c9f-1854-4e6d-b0ed-4fd07076c480 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:10.321837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58179c5-abee-401d-90aa-2e9a96a05dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:10.322439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58179c5-abee-401d-90aa-2e9a96a05dd3 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:10.322720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58179c5-abee-401d-90aa-2e9a96a05dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:10.322993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58179c5-abee-401d-90aa-2e9a96a05dd3 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:10.323642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58179c5-abee-401d-90aa-2e9a96a05dd3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:10.325062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58179c5-abee-401d-90aa-2e9a96a05dd3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:10.350578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 593/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:35:10 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:35:12.268198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe4c-21db-4236-9f5e-ebba1ee97c41 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:12.268747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe4c-21db-4236-9f5e-ebba1ee97c41 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:12.268747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe4c-21db-4236-9f5e-ebba1ee97c41 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:12.270246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 581/1174] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:12.275381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:12.275815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:12.275925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:12.294138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:12.295318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:12.360115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 594/1175] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:35:12.628466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe4c-21db-4236-9f5e-ebba1ee97c41 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:12.715473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:12.729790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe4c-21db-4236-9f5e-ebba1ee97c41 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:12.762476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a5a07-0ba0-4a9e-8bef-65f74868ffec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:12.763095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5a07-0ba0-4a9e-8bef-65f74868ffec None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:12.763374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5a07-0ba0-4a9e-8bef-65f74868ffec None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:12.763645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a5a07-0ba0-4a9e-8bef-65f74868ffec None admin] PATH_INFO: `/v3/domains/6e12820be45f46769100066a9d97aaa0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:12.764750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a5a07-0ba0-4a9e-8bef-65f74868ffec None admin] RBAC: Authorizing `identity:get_domain(domain_id=6e12820be45f46769100066a9d97aaa0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:12.766216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a5a07-0ba0-4a9e-8bef-65f74868ffec None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:12.767749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 582/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Wed Aug 24 05:35:12 2022] GET /identity/v3/domains/6e12820be45f46769100066a9d97aaa0 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:12.773999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:12.792704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:12.873332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d4602-9216-4795-b942-e977bb31b4c3 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:12.906594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06016f9b-d27e-4a93-92f7-27d3c748e9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:12.907129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06016f9b-d27e-4a93-92f7-27d3c748e9d6 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:12.907405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06016f9b-d27e-4a93-92f7-27d3c748e9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:12.907674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06016f9b-d27e-4a93-92f7-27d3c748e9d6 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:12.908351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06016f9b-d27e-4a93-92f7-27d3c748e9d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:12.909795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06016f9b-d27e-4a93-92f7-27d3c748e9d6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:12.931688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 595/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:35:12 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:35:14.870711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fa92a-e1a9-4724-a209-4e5d084f85bb None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:14.871243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fa92a-e1a9-4724-a209-4e5d084f85bb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:14.871243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fa92a-e1a9-4724-a209-4e5d084f85bb None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:14.872587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 583/1178] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:14.878012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:14.878439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:14.878572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:14.896891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:14.898725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:14.963835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 596/1179] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 05:35:15.234476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fa92a-e1a9-4724-a209-4e5d084f85bb None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:15.335130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fa92a-e1a9-4724-a209-4e5d084f85bb None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:15.368907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:15.369522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:15.369829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:15.370097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:15.379824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:35:15.384487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:35:15.385660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:15.386846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:35:15.389719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 584/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:35:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:35:15.391514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:15.394566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:15.486529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9f03b9-2ab0-45db-86f0-d84aea179a32 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:15.519313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:15.519942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:15.520333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:15.520745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:15.521501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:15.521869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:15.523407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:15.531440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 597/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:35:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:15.537365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:15.556919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:15.637483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6757a8d-7f82-450f-814e-9d11c16d21b8 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:15.670797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:15.671413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:15.671691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:15.671963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] PATH_INFO: `/v3/users/e4b38e6c11084e05a721a25aaf1d6865` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:15.684605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b38e6c11084e05a721a25aaf1d6865)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:15.686067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:15.687642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 585/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:35:15 2022] GET /identity/v3/users/e4b38e6c11084e05a721a25aaf1d6865 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:15.693723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:15.756285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:15.790419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7231ef1e-40fd-4601-84c1-79391e53ba23 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:15.821825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:15.822418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:15.822691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:15.822958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:15.887582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:15.889008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:15.951414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:15.984658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 598/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:35:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:35:15.990583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:16.062942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:16.086821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0d6183-f673-4cac-9054-b2832c8a1af6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:16.121542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207dbe8-65aa-43db-b6b9-683a4870fef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:16.122179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dbe8-65aa-43db-b6b9-683a4870fef9 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:16.122456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dbe8-65aa-43db-b6b9-683a4870fef9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:16.122723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207dbe8-65aa-43db-b6b9-683a4870fef9 None admin] PATH_INFO: `/v3/projects/9901eecf1fce49519d71655cbe657c37` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:16.123838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207dbe8-65aa-43db-b6b9-683a4870fef9 None admin] RBAC: Authorizing `identity:get_project(project_id=9901eecf1fce49519d71655cbe657c37)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:16.125255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207dbe8-65aa-43db-b6b9-683a4870fef9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:16.126881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 586/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:35:15 2022] GET /identity/v3/projects/9901eecf1fce49519d71655cbe657c37 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:16.133069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:16.235778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d5f2b-b3c9-46aa-971d-a0484188d476 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:16.269242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:16.270366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:16.270366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:16.270537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:16.271122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=9901eecf1fce49519d71655cbe657c37, user.id=e4b38e6c11084e05a721a25aaf1d6865) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:16.271430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:16.272855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c7d12-984a-4059-87f2-3807dfed3e21 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:16.278291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 599/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:35:16 2022] GET /identity/v3/role_assignments?user.id=e4b38e6c11084e05a721a25aaf1d6865&scope.project.id=9901eecf1fce49519d71655cbe657c37&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:18.144936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af62319-b580-4301-be79-026165c520b2 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:18.145466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af62319-b580-4301-be79-026165c520b2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:18.145466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af62319-b580-4301-be79-026165c520b2 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:18.146851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 587/1186] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:18.152059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:18.152480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:18.152615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:18.170245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:18.171425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:18.195120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:18.237076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 600/1187] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:35:18.502842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af62319-b580-4301-be79-026165c520b2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:18.602140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af62319-b580-4301-be79-026165c520b2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:18.635061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:18.635664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:18.635937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:18.636203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:18.645827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:35:18.650997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:35:18.652088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:35:18.652943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 588/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:35:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:35:18.658229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:18.751623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf4fd8-f74f-4f7c-ba11-366686e301f8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:18.783904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:18.784432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:18.784705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:18.784972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:18.785646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:18.785945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:18.787345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:18.795797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 601/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:35:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:35:18.801750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:18.805658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:18.901410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4e2d0-0950-4694-9570-2c18ec9ac13a None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:18.935256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:18.935855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:18.936134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:18.936404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] PATH_INFO: `/v3/users/e4b38e6c11084e05a721a25aaf1d6865` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:18.937526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] RBAC: Authorizing `identity:get_user(user_id=e4b38e6c11084e05a721a25aaf1d6865)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:18.938978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:18.940529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 589/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:35:18 2022] GET /identity/v3/users/e4b38e6c11084e05a721a25aaf1d6865 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:18.946944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:18.970852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:19.044014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba965e3-939a-455c-b3c3-32e248f51392 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:19.075848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4b162-3906-4832-94b6-67d66d526705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:19.076462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b162-3906-4832-94b6-67d66d526705 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:19.076736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b162-3906-4832-94b6-67d66d526705 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:19.077002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b162-3906-4832-94b6-67d66d526705 None admin] PATH_INFO: `/v3/projects/9901eecf1fce49519d71655cbe657c37` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:19.078113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4b162-3906-4832-94b6-67d66d526705 None admin] RBAC: Authorizing `identity:get_project(project_id=9901eecf1fce49519d71655cbe657c37)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:19.079525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f4b162-3906-4832-94b6-67d66d526705 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:19.081117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 602/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:35:18 2022] GET /identity/v3/projects/9901eecf1fce49519d71655cbe657c37 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:19.087162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:19.154290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:19.185474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e315faa9-7d8b-49db-9e49-717ef57913a7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:19.217823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ccf8e-e01e-4633-a888-15061c1a7f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:19.218465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ccf8e-e01e-4633-a888-15061c1a7f6e None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:19.218747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ccf8e-e01e-4633-a888-15061c1a7f6e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:19.219024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ccf8e-e01e-4633-a888-15061c1a7f6e None admin] PATH_INFO: `/v3/projects/9901eecf1fce49519d71655cbe657c37/users/e4b38e6c11084e05a721a25aaf1d6865/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:19.220890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ccf8e-e01e-4633-a888-15061c1a7f6e None admin] RBAC: Authorizing `identity:create_grant(project_id=9901eecf1fce49519d71655cbe657c37, user_id=e4b38e6c11084e05a721a25aaf1d6865, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:19.222344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13ccf8e-e01e-4633-a888-15061c1a7f6e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:19.236407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 590/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:35:19 2022] PUT /identity/v3/projects/9901eecf1fce49519d71655cbe657c37/users/e4b38e6c11084e05a721a25aaf1d6865/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:35:21.087345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27767b30-7647-4b03-bd4a-c0048f375ed4 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:21.087905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27767b30-7647-4b03-bd4a-c0048f375ed4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:21.087905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27767b30-7647-4b03-bd4a-c0048f375ed4 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:21.089229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 603/1193] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:35:21.094506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:21.094925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:21.095036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:21.112947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:35:21.114159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:35:21.177836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 591/1194] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:35:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:35:21.445462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27767b30-7647-4b03-bd4a-c0048f375ed4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:21.535688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:21.546676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27767b30-7647-4b03-bd4a-c0048f375ed4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:21.580997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:21.581652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:21.581935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:21.582205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:21.592415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:35:21.597142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:35:21.598253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:35:21.599102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 604/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:35:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:35:21.604811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:21.662377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:21.702976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8af4-d73d-4226-8703-f12ef89d768d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:21.735452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:21.736051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:21.736333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:21.736605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:21.737260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:21.737589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:21.738959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:21.747271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 592/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:35:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:21.753028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:21.849583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a556d-a77d-42bc-9a9b-289bf27553fc None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:21.883564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:21.884179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:21.884458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:21.884730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] PATH_INFO: `/v3/users/e4b38e6c11084e05a721a25aaf1d6865` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:21.885904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] RBAC: Authorizing `identity:get_user(user_id=e4b38e6c11084e05a721a25aaf1d6865)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:21.887334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:21.888902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 605/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Wed Aug 24 05:35:21 2022] GET /identity/v3/users/e4b38e6c11084e05a721a25aaf1d6865 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:21.894824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:21.994885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad0173-9219-4634-96b4-4cc45702eb01 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:22.016575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:22.028353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:22.028955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:22.029233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:22.029505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:22.094309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:22.095747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:22.191414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 593/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:35:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:35:22.197421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:22.216474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:22.297670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ea095-630d-48f8-9595-a90b4f1f4d4a None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:22.330115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6078c-c481-445b-8d49-1db695507896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:22.330736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6078c-c481-445b-8d49-1db695507896 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:22.331016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6078c-c481-445b-8d49-1db695507896 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:22.331288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6078c-c481-445b-8d49-1db695507896 None admin] PATH_INFO: `/v3/projects/9901eecf1fce49519d71655cbe657c37` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:22.332394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6078c-c481-445b-8d49-1db695507896 None admin] RBAC: Authorizing `identity:get_project(project_id=9901eecf1fce49519d71655cbe657c37)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:22.333871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6078c-c481-445b-8d49-1db695507896 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:22.335548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 606/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Wed Aug 24 05:35:22 2022] GET /identity/v3/projects/9901eecf1fce49519d71655cbe657c37 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:22.341727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:35:22.365455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:22.439556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25927cd-0ac7-40b3-ab16-b6445c00b903 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:35:22.472121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:35:22.472709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:35:22.472987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:35:22.473254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:35:22.474004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=9901eecf1fce49519d71655cbe657c37, user.id=e4b38e6c11084e05a721a25aaf1d6865) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:35:22.474310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:35:22.476328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ccba5-67d6-4ac0-a01e-2f533a6f2a9a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:35:22.482239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 594/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:35:22 2022] GET /identity/v3/role_assignments?user.id=e4b38e6c11084e05a721a25aaf1d6865&scope.project.id=9901eecf1fce49519d71655cbe657c37&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:35:32.326090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=57 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:35:32.479662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=57 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:18.048749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c278888-6714-4eb7-9889-749d44dc59a2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:18.049364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c278888-6714-4eb7-9889-749d44dc59a2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:18.049456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c278888-6714-4eb7-9889-749d44dc59a2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:18.051147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 607/1201] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:18.058018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:18.058524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:18.058524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:18.077378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:18.078657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:18.157861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 595/1202] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:18.427335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c278888-6714-4eb7-9889-749d44dc59a2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:18.556635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c278888-6714-4eb7-9889-749d44dc59a2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:18.607252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:18.607905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:18.608188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:18.608460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:18.624843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:18.626398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:18.645239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:18.647579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:18.649170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:18.649170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:18.649170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:18.649170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 608/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:18.653453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:18.835839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f363ae3-0c7c-4f75-9ac6-f45a5349c55f None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:18.886090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:18.893767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:18.894416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:18.894739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:18.894964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:18.895617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:18.895915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:18.898132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9731eb80-d9b7-47fc-b8dd-1a79656f7935 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:18.914483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 596/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:36:18.921854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:18.995175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:19.049327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8229b-34f7-4ae2-b12a-204fcefee0f3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:19.094116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcec3f-3394-422b-8187-248c3ca2b806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:19.094972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcec3f-3394-422b-8187-248c3ca2b806 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:19.095416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcec3f-3394-422b-8187-248c3ca2b806 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:19.095829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcec3f-3394-422b-8187-248c3ca2b806 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:19.096802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcec3f-3394-422b-8187-248c3ca2b806 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:19.098570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcec3f-3394-422b-8187-248c3ca2b806 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:19.124303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 609/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Wed Aug 24 05:36:18 2022] POST /identity/v3/users => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:36:21.213928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a6871-81b5-42de-bf05-48ef49111680 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:21.214546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a6871-81b5-42de-bf05-48ef49111680 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:21.214546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a6871-81b5-42de-bf05-48ef49111680 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:21.215840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 597/1206] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:36:21.221165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:21.221665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:21.221768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:21.242141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:21.243803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:21.333603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 610/1207] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:21.601361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a6871-81b5-42de-bf05-48ef49111680 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:21.749385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a6871-81b5-42de-bf05-48ef49111680 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:21.794605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:21.795245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:21.795525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:21.795798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:21.808399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:36:21.813355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:21.814685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:36:21.815875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:21.820355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 598/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:36:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:21.821213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:21.869446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:21.937139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe88c9-c463-41a8-8705-edcdb680c5a8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:21.979397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:21.980012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:21.980291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:21.980565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:21.981242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:21.981546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:21.982990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:21.992736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 611/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:36:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:21.998732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:22.071342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:22.133462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19062b9-1a44-47e7-8978-013ef1f67eec None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:22.179779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:22.180490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:22.180891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:22.181199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:22.194477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:22.196182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:22.211643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:22.212830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:22.213937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:22.213937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:22.213937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:22.218756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:22.219204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 599/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:22.337867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:22.341978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbd1c62-d23c-4b97-bb14-03927d026e63 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:22.380232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:22.381385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:22.381695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:22.382065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:22.382696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:22.383000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:22.384436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:22.401275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 612/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:22.402482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:22.522735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8a078-8e36-4e27-ae97-c54b65202196 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:22.524794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:22.565880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:22.566487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:22.566763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:22.567094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:22.575194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:22.576642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:36:22.589277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:22.590350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:22.591284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 600/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:22.596383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:22.707320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ef081-65b4-4634-a135-aa3c3127d558 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:22.748558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:22.749503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:22.749800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:22.750066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:22.750799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:22.751095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:22.752517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:22.771999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 613/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 05:36:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:22.778096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:22.898036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328dba14-c095-439a-8cf8-041c5f1441d5 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:22.940222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:22.941371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:22.941740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:22.942027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:22.984055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:23.032336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:23.033976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:23.163807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 601/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:36:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:36:23.170016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:23.193244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:23.297273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee475088-b02d-4b42-ba13-7a3b752bf614 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:23.345281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:23.346037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:23.346270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:23.346546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:23.362323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:23.364153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:23.383572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:23.384826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:23.390579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:23.390579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:23.390579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:23.390579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 614/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:23.390872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:23.462006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:23.509972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74515e17-27de-4a2d-8414-49c3a15b6c70 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:23.556823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:23.557247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:23.557444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:23.557807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:23.558593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:23.559014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:23.560754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:23.575062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 602/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:36:23.580625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:23.662620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:23.697393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31141eec-3f6d-412f-a471-0a751ae0a238 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:23.734778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:23.735414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:23.735699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:23.735974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:23.744163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:23.745760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:36:23.758977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:23.759978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:23.765743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 615/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:36:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:23.766432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:23.886432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b08cd-34c4-4c8d-aa9b-6756f88d2744 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:23.909453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:23.924213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:23.924822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:23.925103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:23.925392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:23.926079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:23.926432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:23.928644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f490506-bcbd-476a-befb-e20880929112 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:23.946968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 603/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:36:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:23.949227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:24.062673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e59175-35eb-4fe3-85b9-891f54fa08b0 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:24.097605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:24.105811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:24.106547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:24.106888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:24.107215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:24.108073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=61a7bc1bc86a405a9ef04a3d36502a7e) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:24.108448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:24.110120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ce36d-e0a2-4079-9e59-26d97a98d3b4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:24.116506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 616/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:36:23 2022] GET /identity/v3/role_assignments?user.id=61a7bc1bc86a405a9ef04a3d36502a7e&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:26.206714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17f503-e893-4cfc-919e-abd522df4ebe None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:26.207322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17f503-e893-4cfc-919e-abd522df4ebe None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:26.207322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a17f503-e893-4cfc-919e-abd522df4ebe None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:26.208573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 604/1220] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:26.215055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:26.215524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:26.215629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:26.235329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:26.236580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:26.326182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 617/1221] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:26.606145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17f503-e893-4cfc-919e-abd522df4ebe None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:26.714091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a17f503-e893-4cfc-919e-abd522df4ebe None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:26.752211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:26.752848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:26.753126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:26.753397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:26.763958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:36:26.768815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:26.770332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:26.772013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 605/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:26.776616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:26.885441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25929669-dbcd-4adc-a1cc-056b288d5a06 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:26.916537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:26.930641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:26.931267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:26.931543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:26.931809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:26.932470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] RBAC: Adding query filter params (name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:26.932776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:26.934250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:26.942570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 618/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:36:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:26.949172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:26.997007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:27.059608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eebb2e-f64d-4910-adc3-a9a8aaa534fe None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:27.096702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:27.097335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:27.097647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:27.097924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:27.108581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:27.110103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:27.125471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:27.126661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:27.127593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:27.127593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:27.127593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:27.127593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 606/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:27.133213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:27.237079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb90541-b73f-490e-ac51-49f207e7945b None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:27.285012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:27.285689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:27.286051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:27.286267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:27.286926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:27.287258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:27.288671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:27.301246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 619/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:27.307738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:27.405909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:27.412931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c99281-124c-4cbd-8ddb-b54be98a2004 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:27.449031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:27.449689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:27.449971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:27.450244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:27.457963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:27.459846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:36:27.470995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:27.472570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:27.473936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 607/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:27.478455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:27.529705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:27.595421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045cf28-50b0-48f8-bb4c-c6645fed7e0f None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:27.636160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:27.636777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:27.637055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:27.637329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:27.638088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:27.638394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:27.639878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:27.658510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 620/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 05:36:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:27.664848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:27.768578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0302b0-ec03-4f4b-b6b8-dba8cd171d51 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:27.807570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:27.808201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:27.808593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:27.808929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:27.821059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:27.822612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:27.839717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:27.840718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:27.841652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:27.841652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:27.841652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:27.841652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 608/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:27.847430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:27.965406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8002816-6934-4839-84ee-c3466aa82208 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:27.967863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:28.007597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:28.008284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:28.008563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:28.008831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:28.009503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:28.009875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:28.011426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:28.024976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 621/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:28.030697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:28.042166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:28.152247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d754cc57-21f9-4768-8cbc-f90ce8a9718e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:28.190393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:28.191164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:28.191453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:28.191858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:28.201155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:28.202656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:36:28.215984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:28.216991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:28.218961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 609/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:28.223037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:28.341303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f863030-ce51-4f02-ad85-9c98277eda97 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:28.379961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:28.381245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:28.381521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:28.381986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:28.382710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:28.383137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:28.384911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea05d6-9cc2-4974-aebd-12dfbccd9843 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:28.401782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 622/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:28.408476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:28.489423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:28.528068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447933cf-03fa-4106-8bb8-c56c156d57b4 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:28.569699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c236b437-a4d0-45d6-a7fa-2bb0122d02fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:28.570426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236b437-a4d0-45d6-a7fa-2bb0122d02fb None admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:28.570625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236b437-a4d0-45d6-a7fa-2bb0122d02fb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:28.570903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236b437-a4d0-45d6-a7fa-2bb0122d02fb None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/61a7bc1bc86a405a9ef04a3d36502a7e/roles/beb457731d7b4e8590044c87c6bccfb3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:28.587392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236b437-a4d0-45d6-a7fa-2bb0122d02fb None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=61a7bc1bc86a405a9ef04a3d36502a7e, role_id=beb457731d7b4e8590044c87c6bccfb3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:28.588842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c236b437-a4d0-45d6-a7fa-2bb0122d02fb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:28.602578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 610/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:36:28 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/61a7bc1bc86a405a9ef04a3d36502a7e/roles/beb457731d7b4e8590044c87c6bccfb3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:36:30.647723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac732eb-e357-4c38-91b2-82f1f7ae2fff None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:30.648304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac732eb-e357-4c38-91b2-82f1f7ae2fff None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:30.648304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac732eb-e357-4c38-91b2-82f1f7ae2fff None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:30.649836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 623/1233] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:30.656124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:30.656650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:30.656820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:30.676171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:30.678425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:30.754135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 611/1234] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:31.045040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac732eb-e357-4c38-91b2-82f1f7ae2fff None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:31.071118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:31.157742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac732eb-e357-4c38-91b2-82f1f7ae2fff None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:31.201251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:31.201924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:31.202210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:31.202480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:31.213840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:36:31.218803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:31.219950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:31.220968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 624/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Wed Aug 24 05:36:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:31.226952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:31.284756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:31.343278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381fe1-4f0e-42c9-ba27-4843b1623a01 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:31.382919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:31.383721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:31.384003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:31.384316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:31.385147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] RBAC: Adding query filter params (name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:31.385446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:31.387031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:31.403934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 612/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Wed Aug 24 05:36:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:31.404731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:31.521816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:31.547120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbffb59-6742-4dfd-b72c-399b3f906492 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:31.600549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:31.601086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:31.601364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:31.601665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:31.614230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:31.616538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:31.635945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:31.638136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:31.639938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:31.639938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:31.639938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:31.640269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 625/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:31.647860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:31.768635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c68a9f-d4cb-40eb-a5bc-95e00ae46d7e None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:31.778811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:31.809928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:31.810605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:31.810908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:31.811192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:31.811835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:31.812145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:31.813519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:31.828542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 613/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:36:31.834639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:31.947309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3e4d7-1845-431a-8ef2-cc705d309f58 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:31.994346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:31.995128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:31.995455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:31.995732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:31.999147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:32.010653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:32.012155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:36:32.024729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:32.026133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:32.027744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 626/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:32.032745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:32.138995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3e32e-4069-4b4b-a6f1-edb7c6db80cb None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:32.174538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:32.175148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:32.175425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:32.175705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:32.176431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:32.177007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:32.178750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:32.198538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 614/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 05:36:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:36:32.204437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:32.318884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47862892-b700-44d1-8960-a331abddd9e7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:32.356184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:32.356818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:32.357094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:32.357368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:32.430017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:32.438502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:32.441360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:32.565645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 627/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:36:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:36:32.572594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:32.610342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:32.696522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373212c7-0371-4cee-b627-1fa2db806114 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:32.733855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:32.734632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:32.734872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:32.735169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:32.746046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:32.747489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:32.763656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:32.764682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:32.765607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:32.765607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:32.765607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:32.765607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 615/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:32.771151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:32.875125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:32.882918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc3b4a-e48f-40f1-ba31-44bff5d7bb33 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:32.921542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:32.922269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:32.922544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:32.922888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:32.923602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:32.923909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:32.925476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:32.938827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 628/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:32.945035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:33.036917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:33.060705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adb1b2a-2076-4b35-87ff-db9c25815b19 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:33.101076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:33.101777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:33.102062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:33.102334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:33.110507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:33.112090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:36:33.124734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:33.125919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:33.127146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 616/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:36:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:33.132234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:33.251431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522e92d-0512-44cf-b71d-f50232fa4188 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:33.292708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:33.293350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:33.293693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:33.294003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:33.294699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:33.295006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:33.296641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97839ecc-e6ed-41ae-abb8-aa1f834ec913 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:33.309751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 629/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:36:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:33.316420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:33.425695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df822a7b-be45-4fc8-8160-9c72ee8cb988 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:33.463937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:33.464534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:33.464812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:33.465079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:33.465835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] RBAC: Adding query filter params (role.id=beb457731d7b4e8590044c87c6bccfb3, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=61a7bc1bc86a405a9ef04a3d36502a7e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:33.466146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:33.467598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8174e705-cc32-4876-b975-ccea38bdf3f1 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:33.473833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 617/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:36:33 2022] GET /identity/v3/role_assignments?user.id=61a7bc1bc86a405a9ef04a3d36502a7e&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=beb457731d7b4e8590044c87c6bccfb3 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:36:35.765140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b45e8a-43bc-4752-95cb-662d064580a4 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:35.765761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b45e8a-43bc-4752-95cb-662d064580a4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:35.765761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b45e8a-43bc-4752-95cb-662d064580a4 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:35.767109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 630/1247] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:35.772833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:35.773289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:35.773378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:35.780499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:35.794025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:35.795342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:35.873799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 618/1248] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:36.206454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b45e8a-43bc-4752-95cb-662d064580a4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:36.228822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:36.327061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b45e8a-43bc-4752-95cb-662d064580a4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:36.368526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:36.369361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:36.369822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:36.370132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:36.382715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:36:36.387804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:36.388810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:36.389700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 631/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:36.395506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:36.515089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b0748-7dde-47c6-9a23-348463b4bb1e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:36.551557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:36.559395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:36.560060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:36.560401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:36.560674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:36.561450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:36.561807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:36.563312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:36.573210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 619/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:36:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:36.580888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:36.674338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:36.701859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab9e7e-46f3-4e60-a5c5-b256af4c1418 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:36.738795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:36.739420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:36.739738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:36.740005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:36.753020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:36.754518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:36.771437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:36.772992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:36.774062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:36.774062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:36.774062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:36.774062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 632/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:36.778798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:36.889687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20442a-009f-4ce3-8d1c-5387c49e7a93 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:36.930784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:36.931494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:36.931774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:36.932091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:36.932821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:36.933132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:36.934546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:36.948198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 620/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:36.954232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:37.066783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6edc6ea-bc16-47a1-bf4a-8145d7d58fa6 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:37.100820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:37.112034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:37.113046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:37.113505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:37.114035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:37.125096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:37.127158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:36:37.146898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:37.147922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:37.149016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 633/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:37.154089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:37.218482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:37.272771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73b19b-7ae3-4190-b763-dc97d4df727d None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:37.314522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:37.315140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:37.315447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:37.315715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:37.316761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:37.317238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:37.319396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:37.339127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 621/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 05:36:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:37.344725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:37.466111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63a311-f996-498f-b5a9-db5a80e404e7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:37.508191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:37.508943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:37.509252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:37.509596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:37.576639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:37.593944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:37.595800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:37.713220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 634/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:36:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:36:37.719344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:37.826577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1876d95f-a192-4301-a25a-30c2a31afeff None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:37.841357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:37.865653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:37.866379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:37.866689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:37.867010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:37.877952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:37.879558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:37.897269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:37.898471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:37.899363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:37.899363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:37.899363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:37.899363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 622/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:37.904267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:38.003731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:38.017149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0a4ccc-886a-4e40-88c2-bf655a996d99 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:38.058364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:38.059037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:38.059374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:38.059684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:38.060472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:38.060809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:38.062246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:38.075633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 635/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:38.082264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:38.215070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012daae5-7b3c-4d85-bab2-201f301deca3 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:38.253760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:38.254610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:38.254977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:38.255303 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:38.264221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:38.265711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:36:38.278464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:38.279573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:38.280473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 623/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:36:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:38.288693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:38.414373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bd284-6525-41c4-ab3f-ab884e048b91 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:38.464188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:38.464865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:38.465196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:38.465411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:38.466149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:38.466457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:38.467888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b266454-1003-462e-8c19-161e8a2af888 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:38.482526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 636/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:36:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:38.490817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:38.519879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:38.644032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff605da-b312-4eba-81ff-107860f16139 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:38.682792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:38.683412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:38.683691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:38.683963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:38.684710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=61a7bc1bc86a405a9ef04a3d36502a7e) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:38.685017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:38.686499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bf991-8a24-4042-81ef-403dc0c05a70 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:38.697675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 624/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:36:38 2022] GET /identity/v3/role_assignments?user.id=61a7bc1bc86a405a9ef04a3d36502a7e&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:40.700317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e01fbb-e3f5-44ae-b709-c94b50cea028 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:40.700873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e01fbb-e3f5-44ae-b709-c94b50cea028 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:40.700873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e01fbb-e3f5-44ae-b709-c94b50cea028 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:40.702222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 637/1261] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:40.707285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:40.707704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:40.707838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:40.726222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:40.727404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:40.792864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 625/1262] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:41.064593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e01fbb-e3f5-44ae-b709-c94b50cea028 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:41.075418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:41.165107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e01fbb-e3f5-44ae-b709-c94b50cea028 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:41.197757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:41.198384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:41.198665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:41.198941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:41.209200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:36:41.213932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:41.215013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:41.216052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 638/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:36:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:41.221071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:41.240143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:41.318761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f93f6-4982-4e4f-91e3-4c2388eafeb7 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:41.351694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:41.352296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:41.352575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:41.352848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:41.353798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:41.354101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:41.355467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:41.363250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 626/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:36:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:41.369628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:41.435779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:41.467380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1665b35-dc79-4a22-87d1-c7d54f299b77 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:41.500624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:41.501239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:41.501514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:41.501808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:41.511522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:41.512959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:41.532436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:41.533525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:41.534727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:41.534727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:41.534727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:41.534727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 639/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:41.540199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:41.633971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:41.653294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d695a80e-4120-451c-8687-e3abe6706c7f None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:41.685935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:41.686535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:41.686813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:41.687083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:41.687711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:41.688011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:41.689375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:41.700901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 627/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:41.706612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:41.805734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95b757-a1ed-4c73-a76e-77dfa2bae7a7 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:41.825070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:41.841547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:41.842199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:41.842477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:41.842747 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:41.850658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:41.852075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:36:41.863151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:41.864267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:41.865495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 640/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:41.870340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:41.968729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447fd6b9-2664-4628-804f-ecf5841a0445 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:42.001329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:42.001954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:42.002235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:42.002507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:42.003222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:42.003526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:42.005434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:42.013833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:42.022808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 628/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 05:36:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:42.028469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:42.137524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5009d0aa-b416-4190-ae96-8cb851d8dc01 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:42.169800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:42.170411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:42.170684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:42.170952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:42.180902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:42.182351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:42.196258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:42.197364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:42.198329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:42.198329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:42.198329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:42.198329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 641/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:42.203269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:42.300288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a64f56-346e-4458-9d54-414d34b62cf7 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:42.334680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:42.335272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:42.335548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:42.335815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:42.336442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:42.336740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:42.338129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:42.348932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 629/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:42.354905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:42.386671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:42.452520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953abcf3-ed2c-414c-b9ff-59904daaf8de None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:42.484301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:42.484915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:42.485191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:42.485458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:42.492281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:42.493749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:36:42.504465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:42.506111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:42.507205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 642/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:36:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:42.511938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:42.580249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:42.621417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed98663-b540-4bfc-90f8-b4d268198771 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:42.653737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:42.654356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:42.654640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:42.654906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:42.655586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:42.655892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:42.657314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64521fd-537f-43bd-baa5-b9cd83da1266 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:42.669308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 630/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:36:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:42.675385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:42.764920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:42.773894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff892c-ac01-4508-a457-7306fdeff7e4 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:42.808222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f057e8-d88b-483f-aef6-3d181f0719a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:42.808873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f057e8-d88b-483f-aef6-3d181f0719a9 None admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:42.809156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f057e8-d88b-483f-aef6-3d181f0719a9 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:42.809433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f057e8-d88b-483f-aef6-3d181f0719a9 None admin] PATH_INFO: `/v3/projects/6f233421cc684264987120c578af1f1c/users/61a7bc1bc86a405a9ef04a3d36502a7e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:42.811703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f057e8-d88b-483f-aef6-3d181f0719a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f233421cc684264987120c578af1f1c, user_id=61a7bc1bc86a405a9ef04a3d36502a7e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:42.813273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f057e8-d88b-483f-aef6-3d181f0719a9 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:42.825599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 643/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Wed Aug 24 05:36:42 2022] PUT /identity/v3/projects/6f233421cc684264987120c578af1f1c/users/61a7bc1bc86a405a9ef04a3d36502a7e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:36:44.681074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f656820-0dc2-403f-9f9f-8163c3dca3f8 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:44.681647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f656820-0dc2-403f-9f9f-8163c3dca3f8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:44.681647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f656820-0dc2-403f-9f9f-8163c3dca3f8 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:44.682955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 631/1274] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:44.688016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:44.688450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:44.688543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:44.706637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:44.707794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:44.772344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 644/1275] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:45.036578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f656820-0dc2-403f-9f9f-8163c3dca3f8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:45.136547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f656820-0dc2-403f-9f9f-8163c3dca3f8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:45.153277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:45.170282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:45.170901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.171180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.171450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.182014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 05:36:45.186725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:45.187833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:45.188667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 632/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:45.193818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:45.271451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:45.290437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfffde-473f-47ec-b04b-753526969b17 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:45.323614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:45.324216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.324494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.324764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.325429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:45.325760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:45.327114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:45.335564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 645/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:45.341426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:45.438939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259972c4-80fe-4f06-9cf7-29ba5ec3bc74 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:45.472693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:45.473308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.473663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.474465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.484899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:45.486382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:45.500623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:45.501759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:45.502627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:45.502627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:45.502627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:45.502627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 633/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:45.509874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:45.620785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d61857-f7ce-495d-bed3-adc9269cc4d3 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:45.652975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:45.653596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.653949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.654222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.654854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:45.655156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:45.656508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:45.668346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 646/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:45.674281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:45.704503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:45.775912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492924f-be1e-4f86-8413-2c8b03deb340 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:45.811780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:45.812409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.812686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.812956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.822048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:45.823491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 05:36:45.835015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:45.836182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:45.837327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 634/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:45.841953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:45.845804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:45.938447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bcd0f6-7829-4cf4-9ace-13c0b28ee388 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:45.972065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:45.972669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:45.972939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:45.973208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:45.973951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:45.974256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:45.975671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:45.992377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 647/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:45.998130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:46.086377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:46.095578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ccbec-badd-4d71-98ef-9cf185ab92f6 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:46.129220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:46.129894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.130666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.130935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.197988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:46.199286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:46.286218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:46.296571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 635/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Wed Aug 24 05:36:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 05:36:46.302841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:46.362041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:46.400618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987fd69-2598-41b4-abe8-577330bd8e2d None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:46.434409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:46.435038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.435316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.435588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.445455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:46.446947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 05:36:46.461277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:46.462502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 05:36:46.463592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 05:36:46.463592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 05:36:46.463592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:46.463592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 648/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 05:36:46.468196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:46.574515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7147408-f464-4122-a3be-86dd2f696436 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:46.607866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:46.608463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.608741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.609009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.609662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:46.609967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:46.611328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:46.623220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 636/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Wed Aug 24 05:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:46.629048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:46.725881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fa5d6a-6353-40b2-9de9-a01fd7e4a097 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:46.739014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:46.760027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:46.760678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.760959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.761232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.767694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:46.769139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 05:36:46.780960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:46.782130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: ERROR keystone.server.flask.application Aug 24 05:36:46.783350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 649/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Wed Aug 24 05:36:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:46.788619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:46.838726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:46.888344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488be0b-bb66-4dd3-bea7-340c8db75619 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:46.921337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:46.921960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:46.922240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:46.922517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:46.923195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:46.923500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:46.924918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42161d89-aa4d-49c1-ab22-efc42f3a190a None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:46.936658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 637/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Wed Aug 24 05:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:46.942763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:47.039691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5c5df-11f3-4698-a03e-005804a71c50 None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:47.075260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:47.075861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:47.076140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:47.076410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:47.077142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] RBAC: Adding query filter params (role.id=5ccd18fa93fc4e9f9428ee63d11deba5, scope.project.id=6f233421cc684264987120c578af1f1c, user.id=61a7bc1bc86a405a9ef04a3d36502a7e) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:47.077448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:47.078897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c04498d-ba6b-45f7-8db9-c035f99faccf None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:47.085296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 650/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Wed Aug 24 05:36:46 2022] GET /identity/v3/role_assignments?user.id=61a7bc1bc86a405a9ef04a3d36502a7e&scope.project.id=6f233421cc684264987120c578af1f1c&role.id=5ccd18fa93fc4e9f9428ee63d11deba5 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:48.987630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49daa5b-658f-4cb5-9c0b-03c15f9d8bef None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:48.988158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49daa5b-658f-4cb5-9c0b-03c15f9d8bef None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:48.988274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49daa5b-658f-4cb5-9c0b-03c15f9d8bef None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:48.990161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 638/1288] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:48.995305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:48.995709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:48.995848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:49.013243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:49.014439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:49.043695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:49.079973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 651/1289] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:49.345331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49daa5b-658f-4cb5-9c0b-03c15f9d8bef None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:49.444650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49daa5b-658f-4cb5-9c0b-03c15f9d8bef None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:49.455792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:49.479519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:49.480140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:49.480419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:49.480692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:49.481271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:49.482661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:49.491759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:49.492996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:49.494095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 639/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:36:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:49.498766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:49.609660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e6655-f01e-4753-95c7-aff0ce4bcd1f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:49.643299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:49.643896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:49.644174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:49.644442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:49.645066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:49.645396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:49.646780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1367e8-5d1c-4737-a5cc-432e90a6bdb2 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:49.651391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 652/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 05:36:49 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:49.657133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:49.754414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166b2c2-4236-43bc-8701-b2146089f441 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:49.786914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:49.787509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:49.787784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:49.788051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] PATH_INFO: `/v3/services` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:49.788670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:49.788972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:49.790372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be4a41-3e05-40bd-ba19-fcd3a29c36d0 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:49.794664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 640/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 05:36:49 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:51.663405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634ef83-4e02-456f-a171-64ac8fd805ad None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:51.663955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634ef83-4e02-456f-a171-64ac8fd805ad None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:51.663955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634ef83-4e02-456f-a171-64ac8fd805ad None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:51.665263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 653/1293] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:51.670492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c50c17-5a00-4481-9e9e-3b08dbdd99aa None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:51.671159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c50c17-5a00-4481-9e9e-3b08dbdd99aa None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:51.671159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c50c17-5a00-4481-9e9e-3b08dbdd99aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:51.689356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-12c50c17-5a00-4481-9e9e-3b08dbdd99aa None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:51.690558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-12c50c17-5a00-4481-9e9e-3b08dbdd99aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:51.718390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:51.755808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 641/1294] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:52.020841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634ef83-4e02-456f-a171-64ac8fd805ad None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:52.051132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:52.121701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634ef83-4e02-456f-a171-64ac8fd805ad None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:52.154319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57826b-6825-49e8-a389-f2ee400ddd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:52.154916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57826b-6825-49e8-a389-f2ee400ddd32 None admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:52.155196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57826b-6825-49e8-a389-f2ee400ddd32 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:52.155467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57826b-6825-49e8-a389-f2ee400ddd32 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:52.156101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57826b-6825-49e8-a389-f2ee400ddd32 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:52.157457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57826b-6825-49e8-a389-f2ee400ddd32 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:52.171919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 654/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Wed Aug 24 05:36:52 2022] POST /identity/v3/services => generated 246 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:36:54.066839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3d2a-433e-4f91-bfbf-41277cf39f0c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.067407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3d2a-433e-4f91-bfbf-41277cf39f0c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.067407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bf3d2a-433e-4f91-bfbf-41277cf39f0c None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.068707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 642/1296] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:54.073841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.074251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.074396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.091875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:54.093055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:54.164814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 655/1297] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:54 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:54.429062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3d2a-433e-4f91-bfbf-41277cf39f0c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:54.530255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bf3d2a-433e-4f91-bfbf-41277cf39f0c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:54.569082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:54.569924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.570070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.570260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.570897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:54.572267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:54.581485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:54.583458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:54.584626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 643/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:36:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:54.589764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:54.610974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:54.690412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbea634-daf7-4264-83c9-0e0c345c5037 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:54.723437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:54.724032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.724311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.724580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.725199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] RBAC: Adding query filter params (name=placement) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:54.725499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:54.726899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:54.732119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 656/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 05:36:54 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:36:54.738076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:54.769246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:54.837357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b160e9-6dd7-4a16-bafd-e9596a74cab2 None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:54.871763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:54.872360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:54.872640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:54.872913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:54.873637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] RBAC: Adding query filter params (interface=public, service_id=4dc1edc99e8d44bd91c6b918e8f42a6a, region_id=RegionOne) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:54.873948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:54.875317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b211cc-f793-46e0-a6ab-7936c4499bb5 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:54.880696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 644/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Wed Aug 24 05:36:54 2022] GET /identity/v3/endpoints?service_id=4dc1edc99e8d44bd91c6b918e8f42a6a&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:54.886566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:54.967561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:54.985952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96bab0-964e-4630-82e0-92edb2168cba None admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:55.018580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7828ec79-39f7-4a73-bcfb-b4a0fd0fe1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:55.019179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec79-39f7-4a73-bcfb-b4a0fd0fe1a0 None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:55.019457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec79-39f7-4a73-bcfb-b4a0fd0fe1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:55.019727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828ec79-39f7-4a73-bcfb-b4a0fd0fe1a0 None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:55.020974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828ec79-39f7-4a73-bcfb-b4a0fd0fe1a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:55.022364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7828ec79-39f7-4a73-bcfb-b4a0fd0fe1a0 None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:55.028977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 657/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Wed Aug 24 05:36:54 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:36:56.962731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467affd-5487-4ab5-bdcd-0138d252b1d0 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:56.963256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467affd-5487-4ab5-bdcd-0138d252b1d0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:56.963371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467affd-5487-4ab5-bdcd-0138d252b1d0 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:56.964607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 645/1302] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:36:56.969879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:56.970250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:56.970415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:56.991842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:36:56.993009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:36:57.056852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 658/1303] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:36:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 05:36:57.354376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467affd-5487-4ab5-bdcd-0138d252b1d0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:57.447390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:57.454537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467affd-5487-4ab5-bdcd-0138d252b1d0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:57.488395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f088d-76e6-44b7-9788-6366273061af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:57.489033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f088d-76e6-44b7-9788-6366273061af None admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.489313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f088d-76e6-44b7-9788-6366273061af None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.489606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f088d-76e6-44b7-9788-6366273061af None admin] PATH_INFO: `/v3/services/placement` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.490183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f088d-76e6-44b7-9788-6366273061af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:57.491554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077f088d-76e6-44b7-9788-6366273061af None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application [None req-077f088d-76e6-44b7-9788-6366273061af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 05:36:57.500194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 05:36:57.501325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application with Lock( Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self._enter() Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application return self.creator() Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application created_value = creator( Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: ERROR keystone.server.flask.application Aug 24 05:36:57.502382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 646/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Wed Aug 24 05:36:57 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 05:36:57.507482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:57.581370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:36:57.605894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e29d1-9940-4d10-8303-0faaa16cc74b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:57.638963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:57.639596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.639875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.640143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] PATH_INFO: `/v3/services` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.640760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] RBAC: Adding query filter params (name=placement) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:36:57.641057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:57.642431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a514047-d675-4ce7-998b-39a19826922c None admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:57.647916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 659/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Wed Aug 24 05:36:57 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:36:57.653907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f088d-76e6-44b7-9788-6366273061af None admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:36:57.752917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f088d-76e6-44b7-9788-6366273061af None admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:36:57.787259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b296-1733-4a96-9281-f44b19c7c0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:36:57.787858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b296-1733-4a96-9281-f44b19c7c0f9 None admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:36:57.788136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b296-1733-4a96-9281-f44b19c7c0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:36:57.788405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b296-1733-4a96-9281-f44b19c7c0f9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:36:57.789106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b296-1733-4a96-9281-f44b19c7c0f9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:36:57.790549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b296-1733-4a96-9281-f44b19c7c0f9 None admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:36:57.809659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 647/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:36:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:37:07.645433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=30 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:37:07.805464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:01.976491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c25ac-042e-45b9-832e-6c8cbd824ab9 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:01.977063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c25ac-042e-45b9-832e-6c8cbd824ab9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:01.977063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24c25ac-042e-45b9-832e-6c8cbd824ab9 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:01.982681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 660/1307] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:01.984282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cde741-09f0-4b19-bdaf-deffdbc2a46b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:01.984723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cde741-09f0-4b19-bdaf-deffdbc2a46b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:01.984805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cde741-09f0-4b19-bdaf-deffdbc2a46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:02.083467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d2cde741-09f0-4b19-bdaf-deffdbc2a46b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:02.084794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d2cde741-09f0-4b19-bdaf-deffdbc2a46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.206207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 648/1308] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:38:02.499440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32244d97-e435-4fa0-8216-78620e694f99 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:02.499724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32244d97-e435-4fa0-8216-78620e694f99 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:02.500282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32244d97-e435-4fa0-8216-78620e694f99 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:02.501965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 661/1309] 10.222.0.73 () {54 vars in 965 bytes} [Wed Aug 24 05:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:02.508343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d78ba-a1a7-4b35-ae8e-00676a15c9f4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:02.508635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d78ba-a1a7-4b35-ae8e-00676a15c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:02.508914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d78ba-a1a7-4b35-ae8e-00676a15c9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:02.595392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2a3d78ba-a1a7-4b35-ae8e-00676a15c9f4 None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:02.597285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2a3d78ba-a1a7-4b35-ae8e-00676a15c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.620719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:02.689620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 649/1310] 10.222.0.73 () {58 vars in 1062 bytes} [Wed Aug 24 05:38:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:02.699447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32244d97-e435-4fa0-8216-78620e694f99 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:02.701224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-32244d97-e435-4fa0-8216-78620e694f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.834815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32244d97-e435-4fa0-8216-78620e694f99 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:02.922786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:02.923399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:02.923790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:02.924072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:02.925963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:02.986798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:03.047057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:03.048428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb16c3-b92b-4d44-8203-1abe9da51543 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:03.073595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 662/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:05.337904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14230-ae9e-48e8-bf6e-e59b6f853166 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:05.338419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14230-ae9e-48e8-bf6e-e59b6f853166 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:05.338498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b14230-ae9e-48e8-bf6e-e59b6f853166 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:05.339954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 650/1312] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:05.344889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f732-a723-4bb3-b2f9-99ddc7d4eeea None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:05.345308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f732-a723-4bb3-b2f9-99ddc7d4eeea None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:05.345475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234f732-a723-4bb3-b2f9-99ddc7d4eeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:05.433768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4234f732-a723-4bb3-b2f9-99ddc7d4eeea None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:05.435143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4234f732-a723-4bb3-b2f9-99ddc7d4eeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:05.538405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 663/1313] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:05.818698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b14230-ae9e-48e8-bf6e-e59b6f853166 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:05.820227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-74b14230-ae9e-48e8-bf6e-e59b6f853166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:05.925816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b14230-ae9e-48e8-bf6e-e59b6f853166 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:05.997112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:06.000099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:06.000893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:06.001326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:06.001869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:06.003502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:06.122564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:06.124029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cf9777-64ec-46c3-ac5c-16166f9c8a20 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:06.143725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 651/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:38:08.356897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc5f1d-3739-4904-9212-9761574758cd None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:08.357419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc5f1d-3739-4904-9212-9761574758cd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:08.357419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc5f1d-3739-4904-9212-9761574758cd None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:08.358821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 664/1315] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:08.364558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f871bde-810d-451c-b3a2-d7d674f9324f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:08.364981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f871bde-810d-451c-b3a2-d7d674f9324f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:08.365078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f871bde-810d-451c-b3a2-d7d674f9324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:08.429937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2f871bde-810d-451c-b3a2-d7d674f9324f None None] MFA Rules not processed for user `f3e3708d9b714f0fa7a56007f70bd68e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:08.431127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2f871bde-810d-451c-b3a2-d7d674f9324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:08.482122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:08.526340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 652/1316] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:08.808302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc5f1d-3739-4904-9212-9761574758cd None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:08.809905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ffcc5f1d-3739-4904-9212-9761574758cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:08.820187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:08.915150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc5f1d-3739-4904-9212-9761574758cd None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:08.988564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:08.989170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:08.989448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:08.989744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:08.991357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:09.053394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:09.099081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:09.100510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50078b2d-234d-4f51-90bf-f29998c6e1f1 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:09.119156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 665/1317] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:11.524744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c408084-2662-4f26-a70e-baab0471bac6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:11.525246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c408084-2662-4f26-a70e-baab0471bac6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:11.525246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c408084-2662-4f26-a70e-baab0471bac6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:11.526587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 653/1318] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:11.531826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab471-6081-4106-871b-f11d6316aa29 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:11.532278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab471-6081-4106-871b-f11d6316aa29 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:11.532657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ab471-6081-4106-871b-f11d6316aa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:11.599049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-bf5ab471-6081-4106-871b-f11d6316aa29 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:11.600270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bf5ab471-6081-4106-871b-f11d6316aa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:11.694863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 666/1319] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:11.979885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c408084-2662-4f26-a70e-baab0471bac6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:11.981434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2c408084-2662-4f26-a70e-baab0471bac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:12.084204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c408084-2662-4f26-a70e-baab0471bac6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:12.158807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:12.159439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:12.159721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:12.160505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:12.162134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:12.168975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:12.269925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:12.271294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5e79b8-a765-4725-b917-3f2e0cb67e0a service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:12.289406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 654/1320] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:14.852124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4df9e-c360-4db5-9dcf-7909992ee4c4 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:14.852976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4df9e-c360-4db5-9dcf-7909992ee4c4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:14.852976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4df9e-c360-4db5-9dcf-7909992ee4c4 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:14.854291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 667/1321] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:14.859430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83caff8-586e-47e1-859d-d581dfe49e23 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:14.859862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83caff8-586e-47e1-859d-d581dfe49e23 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:14.860003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83caff8-586e-47e1-859d-d581dfe49e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:14.928278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d83caff8-586e-47e1-859d-d581dfe49e23 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:14.929494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d83caff8-586e-47e1-859d-d581dfe49e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:14.989691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:15.030175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 655/1322] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:15.322067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4df9e-c360-4db5-9dcf-7909992ee4c4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:15.323625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1fb4df9e-c360-4db5-9dcf-7909992ee4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:15.344149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:15.429509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4df9e-c360-4db5-9dcf-7909992ee4c4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:15.503166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:15.503801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:15.504076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:15.504347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:15.505966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:15.578515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:15.617764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:15.619197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168cae92-f98b-4439-b807-1eef5e22b650 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:15.637856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 668/1323] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:17.592824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcb870-81da-4a45-b085-f17149014aa3 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:17.593367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcb870-81da-4a45-b085-f17149014aa3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:17.593367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dcb870-81da-4a45-b085-f17149014aa3 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:17.594737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 656/1324] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:17.600058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168afdf3-68f6-4595-a42e-db26fb64d72d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:17.600847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168afdf3-68f6-4595-a42e-db26fb64d72d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:17.601263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168afdf3-68f6-4595-a42e-db26fb64d72d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:17.667132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-168afdf3-68f6-4595-a42e-db26fb64d72d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:17.668336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-168afdf3-68f6-4595-a42e-db26fb64d72d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:17.763767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 669/1325] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:18.047036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcb870-81da-4a45-b085-f17149014aa3 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:18.048574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-51dcb870-81da-4a45-b085-f17149014aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:18.153553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dcb870-81da-4a45-b085-f17149014aa3 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:18.235747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:18.236352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:18.236629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:18.236900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:18.239113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:18.256114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:18.355309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:18.356762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab53b04-cd14-4c08-9e95-17603ff6412f service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:18.375220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 657/1326] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:20.436217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e7ce-9715-43a3-96aa-463e5633bde4 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:20.436771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e7ce-9715-43a3-96aa-463e5633bde4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:20.436771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c68e7ce-9715-43a3-96aa-463e5633bde4 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:20.438281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 670/1327] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:20.444237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65254-9548-4458-bcef-676bd7d9898b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:20.444651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65254-9548-4458-bcef-676bd7d9898b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:20.444779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a65254-9548-4458-bcef-676bd7d9898b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:20.519148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b6a65254-9548-4458-bcef-676bd7d9898b None None] MFA Rules not processed for user `f3e3708d9b714f0fa7a56007f70bd68e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:20.521112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b6a65254-9548-4458-bcef-676bd7d9898b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:20.588338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:20.622556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 658/1328] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:20.907502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e7ce-9715-43a3-96aa-463e5633bde4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:20.909035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1c68e7ce-9715-43a3-96aa-463e5633bde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:20.940869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:21.014525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c68e7ce-9715-43a3-96aa-463e5633bde4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:21.089297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:21.089943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:21.090223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:21.090495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:21.092109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:21.171589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:21.203533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:21.204976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba8c36-a2e5-42ca-89b9-8782609f30d4 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:21.223310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 671/1329] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:23.381075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c69b8-43c6-44b2-8a2f-6d9d64b8bbdf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:23.381635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c69b8-43c6-44b2-8a2f-6d9d64b8bbdf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:23.381635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c69b8-43c6-44b2-8a2f-6d9d64b8bbdf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:23.382990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 659/1330] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:23.388272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86854305-a2b5-4609-a876-dcbb7db64006 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:23.388666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86854305-a2b5-4609-a876-dcbb7db64006 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:23.388830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86854305-a2b5-4609-a876-dcbb7db64006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:23.456122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-86854305-a2b5-4609-a876-dcbb7db64006 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:23.457948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-86854305-a2b5-4609-a876-dcbb7db64006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:23.559134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 672/1331] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:23.840261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c69b8-43c6-44b2-8a2f-6d9d64b8bbdf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:23.841826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6e8c69b8-43c6-44b2-8a2f-6d9d64b8bbdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:23.944679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c69b8-43c6-44b2-8a2f-6d9d64b8bbdf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:24.016904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:24.017503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:24.017810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:24.018085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:24.020263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:24.047330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:24.129348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:24.130765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a405667-65eb-441b-bbf4-bb00971560db service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:24.150550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 660/1332] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:26.072973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679d2ca-a328-437f-80a7-accadfee0200 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:26.073535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679d2ca-a328-437f-80a7-accadfee0200 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:26.073535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b679d2ca-a328-437f-80a7-accadfee0200 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:26.074905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 673/1333] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:26.080337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6daa73-9c03-4b49-8687-3acc93a491b2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:26.080783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6daa73-9c03-4b49-8687-3acc93a491b2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:26.080888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6daa73-9c03-4b49-8687-3acc93a491b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:26.145610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-4b6daa73-9c03-4b49-8687-3acc93a491b2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:26.146808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4b6daa73-9c03-4b49-8687-3acc93a491b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:26.223297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:26.246390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 661/1334] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:38:26.527061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679d2ca-a328-437f-80a7-accadfee0200 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:26.528646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b679d2ca-a328-437f-80a7-accadfee0200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:26.568856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:26.632677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b679d2ca-a328-437f-80a7-accadfee0200 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:26.704919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:26.705514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:26.705830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:26.706598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:26.708362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:26.790350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:26.815899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:26.817316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a2ee2f-133f-47e6-8c2c-95cbd49092e9 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:26.835064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 674/1335] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:29.099914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcc3b8-1b27-4c87-8583-8fc150db76e0 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:29.100443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcc3b8-1b27-4c87-8583-8fc150db76e0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:29.100556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bcc3b8-1b27-4c87-8583-8fc150db76e0 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:29.101832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 662/1336] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:29.107266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f70fd-3305-414f-95a8-f229c6411f96 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:29.107668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f70fd-3305-414f-95a8-f229c6411f96 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:29.107805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411f70fd-3305-414f-95a8-f229c6411f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:29.171752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-411f70fd-3305-414f-95a8-f229c6411f96 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:29.172933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-411f70fd-3305-414f-95a8-f229c6411f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:29.267964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 675/1337] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:29.570309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1164c-580d-46ba-a13f-bd3077cd91a5 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:29.570594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1164c-580d-46ba-a13f-bd3077cd91a5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:29.570874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1164c-580d-46ba-a13f-bd3077cd91a5 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:29.572209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 663/1338] 10.222.0.73 () {54 vars in 965 bytes} [Wed Aug 24 05:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:29.578036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463f35f-b17d-4012-8da9-087d67672b1f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:29.578323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463f35f-b17d-4012-8da9-087d67672b1f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:29.578596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463f35f-b17d-4012-8da9-087d67672b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:29.632638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:29.653738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c463f35f-b17d-4012-8da9-087d67672b1f None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:29.654950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c463f35f-b17d-4012-8da9-087d67672b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:29.729950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 676/1339] 10.222.0.73 () {58 vars in 1062 bytes} [Wed Aug 24 05:38:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:29.739303 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1164c-580d-46ba-a13f-bd3077cd91a5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:29.740811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b7f1164c-580d-46ba-a13f-bd3077cd91a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:29.842333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1164c-580d-46ba-a13f-bd3077cd91a5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:29.914386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:29.914981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:29.915257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:29.915526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:29.917110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:29.952489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:30.024639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:30.026022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bec908-3bce-4331-927b-54b7228c879a service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:30.043939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 664/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:34.429272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bc9d4-a71a-4d53-ab0e-d5dbf6950770 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:34.429873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bc9d4-a71a-4d53-ab0e-d5dbf6950770 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:34.429873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8bc9d4-a71a-4d53-ab0e-d5dbf6950770 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:34.431228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 677/1341] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:34.436713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588fbbd-9b42-463c-836e-1b542502f32f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:34.437113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588fbbd-9b42-463c-836e-1b542502f32f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:34.437304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588fbbd-9b42-463c-836e-1b542502f32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:34.506070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-9588fbbd-9b42-463c-836e-1b542502f32f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:34.507286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9588fbbd-9b42-463c-836e-1b542502f32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:34.595711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:34.608252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 665/1342] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:34.919898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bc9d4-a71a-4d53-ab0e-d5dbf6950770 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:34.921599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aa8bc9d4-a71a-4d53-ab0e-d5dbf6950770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:35.026674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8bc9d4-a71a-4d53-ab0e-d5dbf6950770 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:35.066837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:35.105366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:35.105999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:35.106273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:35.106534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:35.108905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:35.234138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:35.235474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1066522-8fd2-4851-b2d7-f6f01cfca7a5 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:35.255882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 678/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:37.817684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b19c-5389-48c9-ad20-a37413e30871 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:37.818230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b19c-5389-48c9-ad20-a37413e30871 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:37.818230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009b19c-5389-48c9-ad20-a37413e30871 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:37.819553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 666/1344] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:37.824719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67610296-9abd-4f01-9e1c-43434d9af98a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:37.825134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67610296-9abd-4f01-9e1c-43434d9af98a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:37.825267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67610296-9abd-4f01-9e1c-43434d9af98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:37.892479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-67610296-9abd-4f01-9e1c-43434d9af98a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:37.893837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-67610296-9abd-4f01-9e1c-43434d9af98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:37.911990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:37.991074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 679/1345] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:38.277719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b19c-5389-48c9-ad20-a37413e30871 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:38.279238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6009b19c-5389-48c9-ad20-a37413e30871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:38.385493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009b19c-5389-48c9-ad20-a37413e30871 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:38.463433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:38.464039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:38.464321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:38.464594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:38.466210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:38.510688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:38.583894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:38.585342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6314cf-6763-4651-a59c-7e7f244635c0 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:38.604231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 667/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:42.464776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d707abc-2ecd-4032-a456-223feb755762 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:42.465225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d707abc-2ecd-4032-a456-223feb755762 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:42.465405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d707abc-2ecd-4032-a456-223feb755762 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:42.466703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 680/1347] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:42.473695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c6b2-d0b4-411f-af82-5786927aeebe None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:42.474102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c6b2-d0b4-411f-af82-5786927aeebe None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:42.475527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113c6b2-d0b4-411f-af82-5786927aeebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:42.557245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0113c6b2-d0b4-411f-af82-5786927aeebe None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:42.558504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0113c6b2-d0b4-411f-af82-5786927aeebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:42.671759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:42.676591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 668/1348] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:42.958314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d707abc-2ecd-4032-a456-223feb755762 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:42.959873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3d707abc-2ecd-4032-a456-223feb755762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:43.071378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d707abc-2ecd-4032-a456-223feb755762 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:43.121025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:43.148703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:43.149311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:43.149612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:43.149885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:43.151505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:43.262517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:43.263993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8a1ba2-1bc0-4dd9-9408-bc3b7d53203c service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:43.281720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 681/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:38:48.208140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27e12-89c8-4514-a1ac-ad86810752e5 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:48.208140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27e12-89c8-4514-a1ac-ad86810752e5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:48.208140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e27e12-89c8-4514-a1ac-ad86810752e5 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:48.215055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 669/1350] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:38:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:48.223547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645e81b-6dc1-4058-b9a6-017ee654cc30 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:48.223547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645e81b-6dc1-4058-b9a6-017ee654cc30 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:48.223547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645e81b-6dc1-4058-b9a6-017ee654cc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:48.313203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-7645e81b-6dc1-4058-b9a6-017ee654cc30 None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:48.314544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7645e81b-6dc1-4058-b9a6-017ee654cc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:48.347671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:48.471893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 682/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:38:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:52.843343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385f60-690d-4748-9f8d-e91b15188e98 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:52.844015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385f60-690d-4748-9f8d-e91b15188e98 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:52.844015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c385f60-690d-4748-9f8d-e91b15188e98 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:52.845224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 670/1352] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:38:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:52.850936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edab916-f285-4826-9984-f466b42204a3 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:52.851477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edab916-f285-4826-9984-f466b42204a3 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:52.851892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edab916-f285-4826-9984-f466b42204a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:52.942542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-6edab916-f285-4826-9984-f466b42204a3 None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:52.943757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6edab916-f285-4826-9984-f466b42204a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:53.029949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:53.074934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 683/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:38:52 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:54.220600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36ed4-b89c-4984-b250-43052660e35d None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:54.221611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36ed4-b89c-4984-b250-43052660e35d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:54.221611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d36ed4-b89c-4984-b250-43052660e35d None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:54.223144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 671/1354] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:54.228131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cfdc3-b02d-43a4-9af9-f475b0a0c8f4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:54.228575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cfdc3-b02d-43a4-9af9-f475b0a0c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:54.228813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42cfdc3-b02d-43a4-9af9-f475b0a0c8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:54.308595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b42cfdc3-b02d-43a4-9af9-f475b0a0c8f4 None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:54.309822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b42cfdc3-b02d-43a4-9af9-f475b0a0c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:54.418705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 684/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:38:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:38:57.708016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d47d-40e9-411c-9ebf-4c73c8bffd31 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:57.708554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d47d-40e9-411c-9ebf-4c73c8bffd31 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:57.708652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4388d47d-40e9-411c-9ebf-4c73c8bffd31 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:57.709960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 672/1356] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:57.715393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeeddbe-168f-468e-b763-4ef0b1d6b853 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:57.715818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeeddbe-168f-468e-b763-4ef0b1d6b853 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:57.716071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeeddbe-168f-468e-b763-4ef0b1d6b853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:57.767445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:57.823370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1aeeddbe-168f-468e-b763-4ef0b1d6b853 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:57.824562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1aeeddbe-168f-468e-b763-4ef0b1d6b853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:57.965710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 685/1357] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:38:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:58.280417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20f29e-b3b9-4aee-a920-46ee287a13c0 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:58.280417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20f29e-b3b9-4aee-a920-46ee287a13c0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:58.280601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a20f29e-b3b9-4aee-a920-46ee287a13c0 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:58.282339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 673/1358] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 05:38:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:38:58.290399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19a21-05bc-4d99-bbfd-8c2a87331c58 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:58.290969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19a21-05bc-4d99-bbfd-8c2a87331c58 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:58.291452 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19a21-05bc-4d99-bbfd-8c2a87331c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:58.383463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3cb19a21-05bc-4d99-bbfd-8c2a87331c58 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:38:58.384790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3cb19a21-05bc-4d99-bbfd-8c2a87331c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:58.436601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:58.508536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 686/1359] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 05:38:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:38:58.516226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20f29e-b3b9-4aee-a920-46ee287a13c0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:38:58.518092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2a20f29e-b3b9-4aee-a920-46ee287a13c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:58.674824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a20f29e-b3b9-4aee-a920-46ee287a13c0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:38:58.757218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:38:58.804841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:38:58.805451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:38:58.806394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:38:58.806676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:38:58.808271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:38:58.904292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:38:58.906151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4f1b4-f2fe-43eb-a6ca-7cafc8b0d58f service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:38:58.926545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 674/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:38:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:39:01.087573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27a83-fc92-4bb3-ac26-b1f8c893d664 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:01.088098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27a83-fc92-4bb3-ac26-b1f8c893d664 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:01.088098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d27a83-fc92-4bb3-ac26-b1f8c893d664 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:01.089428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 687/1361] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:01.095200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708125-452a-43cb-8255-c39cf152b8ad None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:01.096498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708125-452a-43cb-8255-c39cf152b8ad None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:01.096781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76708125-452a-43cb-8255-c39cf152b8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:01.167425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-76708125-452a-43cb-8255-c39cf152b8ad None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:01.168625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-76708125-452a-43cb-8255-c39cf152b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:01.208377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:01.284249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 675/1362] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:01.575296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d27a83-fc92-4bb3-ac26-b1f8c893d664 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:01.576732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-56d27a83-fc92-4bb3-ac26-b1f8c893d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:01.741321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d27a83-fc92-4bb3-ac26-b1f8c893d664 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:01.751966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:01.871109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:01.871724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:01.872010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:01.872281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:01.873910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:01.978577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:01.980240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b956ca-09ac-4835-ad02-daeed5a0ad0e service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:01.999625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 688/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:04.249698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f506-a1c6-42ab-8e5a-385db04a53fb None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:04.250254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f506-a1c6-42ab-8e5a-385db04a53fb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:04.250254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c33f506-a1c6-42ab-8e5a-385db04a53fb None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:04.251611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 676/1364] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:39:04.257318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13a66c-7016-4d29-a9f7-6ad65a9a54f8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:04.258150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13a66c-7016-4d29-a9f7-6ad65a9a54f8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:04.258608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13a66c-7016-4d29-a9f7-6ad65a9a54f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:04.277786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:04.332461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3b13a66c-7016-4d29-a9f7-6ad65a9a54f8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:04.333738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3b13a66c-7016-4d29-a9f7-6ad65a9a54f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:04.440849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 689/1365] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:39:04.737536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e738-7c17-4d54-bcfe-9efcf27276ea None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:04.737986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e738-7c17-4d54-bcfe-9efcf27276ea None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:04.738375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627e738-7c17-4d54-bcfe-9efcf27276ea None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:04.740023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 677/1366] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 05:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:04.745799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2044-6b72-4030-875a-1d6132e1915a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:04.746193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2044-6b72-4030-875a-1d6132e1915a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:04.746586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b2044-6b72-4030-875a-1d6132e1915a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:04.836177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-164b2044-6b72-4030-875a-1d6132e1915a None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:04.837543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-164b2044-6b72-4030-875a-1d6132e1915a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:04.877888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:04.969334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 690/1367] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 05:39:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:04.977886 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627e738-7c17-4d54-bcfe-9efcf27276ea None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:04.979511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d627e738-7c17-4d54-bcfe-9efcf27276ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:05.149511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627e738-7c17-4d54-bcfe-9efcf27276ea None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:05.171987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:05.281486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:05.282129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:05.282380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:05.282647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:05.284706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:05.384551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:05.386485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9eaa3a-e6f7-4aa7-b331-ef68df2da42d service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:05.405667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 678/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:05.944618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81736168-d708-4520-bc9d-17be5ad3d478 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:05.945151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81736168-d708-4520-bc9d-17be5ad3d478 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:05.945151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81736168-d708-4520-bc9d-17be5ad3d478 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:05.946706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 691/1369] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:05.951833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ded7fe-a53f-4787-8bc5-9d2628d37596 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:05.952126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ded7fe-a53f-4787-8bc5-9d2628d37596 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:05.952399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ded7fe-a53f-4787-8bc5-9d2628d37596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:05.988480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:06.024848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-30ded7fe-a53f-4787-8bc5-9d2628d37596 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:06.026101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-30ded7fe-a53f-4787-8bc5-9d2628d37596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:06.152594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 679/1370] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:06.765546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af30d72-f65d-4199-b5a6-57e1be8d08e9 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:06.765836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af30d72-f65d-4199-b5a6-57e1be8d08e9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:06.766079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af30d72-f65d-4199-b5a6-57e1be8d08e9 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:06.767599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 692/1371] 10.222.0.73 () {54 vars in 964 bytes} [Wed Aug 24 05:39:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:06.773286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fc868-9f57-4778-8a8b-8c3c6c40fa68 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:06.773790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fc868-9f57-4778-8a8b-8c3c6c40fa68 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:06.774159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fc868-9f57-4778-8a8b-8c3c6c40fa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:06.863012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a39fc868-9f57-4778-8a8b-8c3c6c40fa68 None None] MFA Rules not processed for user `1c7f12a5dc6446eaa94a96dbb44ccdaf`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:06.864536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a39fc868-9f57-4778-8a8b-8c3c6c40fa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:06.915522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:06.950245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 680/1372] 10.222.0.73 () {58 vars in 1061 bytes} [Wed Aug 24 05:39:06 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:39:06.959278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af30d72-f65d-4199-b5a6-57e1be8d08e9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:06.960880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0af30d72-f65d-4199-b5a6-57e1be8d08e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:07.081708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af30d72-f65d-4199-b5a6-57e1be8d08e9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:07.146229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:07.194171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:07.194951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:07.195303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:07.195699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:07.197705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:07.340648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:07.342250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caacea25-beb3-4024-a113-d3a249878832 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:07.367146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 693/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:07.415986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc07c2b-b670-4393-b68c-d12d331ff3c5 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:07.416698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc07c2b-b670-4393-b68c-d12d331ff3c5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:07.416698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc07c2b-b670-4393-b68c-d12d331ff3c5 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:07.417960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 681/1374] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:07.423024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a805-1241-461b-bb01-52835c64bd50 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:07.423335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a805-1241-461b-bb01-52835c64bd50 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:07.424147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a805-1241-461b-bb01-52835c64bd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:07.619138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-bf46a805-1241-461b-bb01-52835c64bd50 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:07.620369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bf46a805-1241-461b-bb01-52835c64bd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:07.642736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:07.719686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 694/1375] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:08.009890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc07c2b-b670-4393-b68c-d12d331ff3c5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:08.011837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0cc07c2b-b670-4393-b68c-d12d331ff3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:08.164195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc07c2b-b670-4393-b68c-d12d331ff3c5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:08.215300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:08.280100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:08.280623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:08.280908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:08.281181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:08.283324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:08.375420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:08.377057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd46c97-8f50-407e-9925-6202df10ef38 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:08.395542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 682/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:10.337144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db7b6a-1bad-4ba3-8c10-14f7af6cc776 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:10.337668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db7b6a-1bad-4ba3-8c10-14f7af6cc776 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:10.337766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db7b6a-1bad-4ba3-8c10-14f7af6cc776 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:10.339046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 695/1377] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:10.344321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b00889-0039-4fcc-8550-47dc55f7981c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:10.344746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b00889-0039-4fcc-8550-47dc55f7981c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:10.344853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b00889-0039-4fcc-8550-47dc55f7981c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:10.417537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b8b00889-0039-4fcc-8550-47dc55f7981c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:10.418832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b8b00889-0039-4fcc-8550-47dc55f7981c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:10.430928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:10.547252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 683/1378] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:10.840115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db7b6a-1bad-4ba3-8c10-14f7af6cc776 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:10.846063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-75db7b6a-1bad-4ba3-8c10-14f7af6cc776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:11.045444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db7b6a-1bad-4ba3-8c10-14f7af6cc776 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:11.055831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:11.191101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:11.191622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:11.191894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:11.192161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:11.194087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:11.299090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:11.300726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf232d1-4c15-432d-a46b-ca001380cad9 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:11.319142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 696/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:13.296891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258e6c-bfbc-43f0-8730-ddd299982a5a None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:13.296891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258e6c-bfbc-43f0-8730-ddd299982a5a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:13.297686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33258e6c-bfbc-43f0-8730-ddd299982a5a None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:13.303781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fcd5a-636e-4bb2-a584-353a74005863 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:13.304224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fcd5a-636e-4bb2-a584-353a74005863 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:13.304648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58fcd5a-636e-4bb2-a584-353a74005863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:13.308592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 684/1380] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:13.316200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:13.375597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a58fcd5a-636e-4bb2-a584-353a74005863 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:13.377349 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a58fcd5a-636e-4bb2-a584-353a74005863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:13.481681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 697/1381] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:39:13.793144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258e6c-bfbc-43f0-8730-ddd299982a5a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:13.795172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-33258e6c-bfbc-43f0-8730-ddd299982a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:13.956715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:13.968400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33258e6c-bfbc-43f0-8730-ddd299982a5a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:14.095616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:14.096231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:14.096556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:14.096835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:14.098496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:14.194364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:14.196006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d5e791-3720-47fe-aeeb-54b6537d52bb service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:14.206827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:14.216135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 685/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:16.183262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b551-d188-401e-b56d-54d37abf2135 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:16.184009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b551-d188-401e-b56d-54d37abf2135 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:16.184009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c9b551-d188-401e-b56d-54d37abf2135 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:16.185152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 698/1383] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:39:16.190582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ce18-a767-4e44-a2e5-faa4c18a165a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:16.191183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ce18-a767-4e44-a2e5-faa4c18a165a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:16.191183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467ce18-a767-4e44-a2e5-faa4c18a165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:16.276205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e467ce18-a767-4e44-a2e5-faa4c18a165a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:16.277838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e467ce18-a767-4e44-a2e5-faa4c18a165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:16.384613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 686/1384] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:16.687529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9b551-d188-401e-b56d-54d37abf2135 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:16.689687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-79c9b551-d188-401e-b56d-54d37abf2135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:16.781640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:16.845945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c9b551-d188-401e-b56d-54d37abf2135 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:16.980504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:16.981027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:16.981306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:16.981602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:16.983238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:17.034969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:17.076551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:17.078210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae30735-c99a-4984-ade5-c96acb3bfe31 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:17.097733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 699/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:19.024044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3e7c-970c-46aa-b3c1-bd222a7fe523 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:19.024583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3e7c-970c-46aa-b3c1-bd222a7fe523 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:19.024583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f3e7c-970c-46aa-b3c1-bd222a7fe523 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:19.025973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 687/1386] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:39:19.031965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b2a0-9ac7-4fb2-a298-f6a9129e5e63 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:19.032433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b2a0-9ac7-4fb2-a298-f6a9129e5e63 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:19.032780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78b2a0-9ac7-4fb2-a298-f6a9129e5e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:19.117444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5c78b2a0-9ac7-4fb2-a298-f6a9129e5e63 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:19.119082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5c78b2a0-9ac7-4fb2-a298-f6a9129e5e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:19.217214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 700/1387] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:19.505532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3e7c-970c-46aa-b3c1-bd222a7fe523 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:19.507246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-488f3e7c-970c-46aa-b3c1-bd222a7fe523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:19.597760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:19.671307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f3e7c-970c-46aa-b3c1-bd222a7fe523 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:19.794682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:19.795308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:19.795569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:19.795838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:19.797433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:19.846429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:19.903931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:19.905530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471a11d0-f944-4e4c-b188-2fe42527c7dc service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:19.924385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 688/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:21.983249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa3ee0-9441-49d2-817b-66ac76d91472 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:21.983811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa3ee0-9441-49d2-817b-66ac76d91472 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:21.983811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa3ee0-9441-49d2-817b-66ac76d91472 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:21.985133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 701/1389] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:39:21.990970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb154762-88bc-434f-ac62-220a53926038 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:21.991414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb154762-88bc-434f-ac62-220a53926038 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:21.991505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb154762-88bc-434f-ac62-220a53926038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:22.058407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-fb154762-88bc-434f-ac62-220a53926038 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:22.059611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fb154762-88bc-434f-ac62-220a53926038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:22.159206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 689/1390] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:22.454569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa3ee0-9441-49d2-817b-66ac76d91472 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:22.456112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7afa3ee0-9441-49d2-817b-66ac76d91472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:22.488976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:22.489273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1b5e-2c14-47cf-b148-5c7d969f25cb None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:22.489603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1b5e-2c14-47cf-b148-5c7d969f25cb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:22.489957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b1b5e-2c14-47cf-b148-5c7d969f25cb None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:22.491472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 690/1391] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:39:22.496569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbf28b-f543-4123-b1a4-d09733741fb3 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:22.496865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbf28b-f543-4123-b1a4-d09733741fb3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:22.497146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbf28b-f543-4123-b1a4-d09733741fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:22.514569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:22.570089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-27bbf28b-f543-4123-b1a4-d09733741fb3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:22.571341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-27bbf28b-f543-4123-b1a4-d09733741fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:22.613485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa3ee0-9441-49d2-817b-66ac76d91472 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:22.671812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 691/1392] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:39:22.716998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:22.730300 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:22.730904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:22.731180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:22.731449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:22.733026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:22.825009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:22.826555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab5b1f9-65a1-44af-afd7-31c4149c384e service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:22.846233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 702/1393] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:22.973581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e1c9-0633-4612-8841-dd46cde1d8c0 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:22.973868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e1c9-0633-4612-8841-dd46cde1d8c0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:22.974146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5e1c9-0633-4612-8841-dd46cde1d8c0 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:22.975498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 692/1394] 10.222.0.73 () {54 vars in 967 bytes} [Wed Aug 24 05:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:22.981347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080eda7-2f83-4e81-b8a4-6ccea5e71dc6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:22.981656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080eda7-2f83-4e81-b8a4-6ccea5e71dc6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:22.981934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e080eda7-2f83-4e81-b8a4-6ccea5e71dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:23.065718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e080eda7-2f83-4e81-b8a4-6ccea5e71dc6 None None] MFA Rules not processed for user `960468d880f34ea2a267b6ada5fb30bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:23.066938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e080eda7-2f83-4e81-b8a4-6ccea5e71dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:23.119814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:23.144872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 703/1395] 10.222.0.73 () {58 vars in 1064 bytes} [Wed Aug 24 05:39:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:23.152373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e1c9-0633-4612-8841-dd46cde1d8c0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:23.153921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3ed5e1c9-0633-4612-8841-dd46cde1d8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:23.252803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:23.260083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed5e1c9-0633-4612-8841-dd46cde1d8c0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:23.336429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:23.337055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:23.337325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:23.337620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:23.339224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:23.450813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:23.452184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf940f-fcff-4ee3-9522-60f3c56198d9 service glance] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:23.471110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 693/1396] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:24.210125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548ca29-ea05-4a1b-a7c4-7cf2b7277431 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:24.210606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548ca29-ea05-4a1b-a7c4-7cf2b7277431 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:24.210698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548ca29-ea05-4a1b-a7c4-7cf2b7277431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:24.279305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9548ca29-ea05-4a1b-a7c4-7cf2b7277431 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:24.280525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9548ca29-ea05-4a1b-a7c4-7cf2b7277431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:24.367046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 704/1397] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:39:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:24.415347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d55098-426f-4eac-9e46-a8f36aa000af None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:24.415835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d55098-426f-4eac-9e46-a8f36aa000af None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:24.416022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d55098-426f-4eac-9e46-a8f36aa000af None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:24.417420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 694/1398] 10.222.0.73 () {54 vars in 943 bytes} [Wed Aug 24 05:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:24.424902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b2a41-18b5-4946-810d-0f6198087a1d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:24.425380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b2a41-18b5-4946-810d-0f6198087a1d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:24.425696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09b2a41-18b5-4946-810d-0f6198087a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:24.503003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f09b2a41-18b5-4946-810d-0f6198087a1d None None] MFA Rules not processed for user `cdca2e846d9340a5ab6ee8356115a6c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:24.504501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f09b2a41-18b5-4946-810d-0f6198087a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:24.517309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:24.615272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 705/1399] 10.222.0.73 () {58 vars in 1040 bytes} [Wed Aug 24 05:39:24 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:24.624613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d55098-426f-4eac-9e46-a8f36aa000af None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:24.626827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d7d55098-426f-4eac-9e46-a8f36aa000af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:24.651867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:24.763856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d55098-426f-4eac-9e46-a8f36aa000af None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:24.798779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca9fd-1c1c-492c-ac5e-7404ff8de2f9 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:24.799062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca9fd-1c1c-492c-ac5e-7404ff8de2f9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:24.799330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30ca9fd-1c1c-492c-ac5e-7404ff8de2f9 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:24.800674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 706/1400] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:24.805663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f5e9-a215-4fb7-8504-c349ba45ac41 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:24.805949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f5e9-a215-4fb7-8504-c349ba45ac41 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:24.806227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165f5e9-a215-4fb7-8504-c349ba45ac41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:24.874208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b165f5e9-a215-4fb7-8504-c349ba45ac41 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:24.875441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b165f5e9-a215-4fb7-8504-c349ba45ac41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:24.879199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:24.879819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:24.880343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:24.880615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:24.882337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:24.889915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:24.971189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:24.983742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:24.986106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:24.995730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 695/1401] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:39:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:39:25.001194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 707/1402] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:39:25.306535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:25.307744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:25.413651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ad082-fa4f-4e63-9976-0061ad8f5612 service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:25.472746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:25.536330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:25.537053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:25.537404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:25.537718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:25.539595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:25.637064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:25.638723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a376b2-1cf6-4cfd-9fb6-5a0a0cb3b582 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:25.659057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 696/1403] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:39:27.590625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e10c-0d06-488d-9481-47bd05e92bc2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:27.591237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e10c-0d06-488d-9481-47bd05e92bc2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:27.591237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f807e10c-0d06-488d-9481-47bd05e92bc2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:27.592787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 708/1404] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:27.598791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca41597-643d-441e-a45c-2cd7ffdf6eb6 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:27.599267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca41597-643d-441e-a45c-2cd7ffdf6eb6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:27.599372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca41597-643d-441e-a45c-2cd7ffdf6eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:27.678222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-cca41597-643d-441e-a45c-2cd7ffdf6eb6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:27.680347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cca41597-643d-441e-a45c-2cd7ffdf6eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:27.695077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:27.792134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 697/1405] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:28.081548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807e10c-0d06-488d-9481-47bd05e92bc2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:28.083102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f807e10c-0d06-488d-9481-47bd05e92bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:28.259101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f807e10c-0d06-488d-9481-47bd05e92bc2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:28.327395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:28.391844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:28.392454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:28.392734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:28.393005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:28.395736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:28.501250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:28.502895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5acf38-db56-411c-b75c-872f3b7ed2f1 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:28.521519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 709/1406] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:30.580837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b0bb-7b16-4963-8585-68cf0095efe6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:30.581425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b0bb-7b16-4963-8585-68cf0095efe6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:30.581425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d2b0bb-7b16-4963-8585-68cf0095efe6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:30.582984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 698/1407] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:30.589116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644ba1-7ec7-4314-858a-172729ed5015 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:30.589700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644ba1-7ec7-4314-858a-172729ed5015 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:30.589700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644ba1-7ec7-4314-858a-172729ed5015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:30.674548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2e644ba1-7ec7-4314-858a-172729ed5015 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:30.678158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2e644ba1-7ec7-4314-858a-172729ed5015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:30.698943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:30.788497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 710/1408] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:39:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:31.086724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2b0bb-7b16-4963-8585-68cf0095efe6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:31.088458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d9d2b0bb-7b16-4963-8585-68cf0095efe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:31.255960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:31.259845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d2b0bb-7b16-4963-8585-68cf0095efe6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:31.399664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:31.400264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:31.400544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:31.400821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:31.402461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:31.503242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:31.504976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a29a4-3f16-4098-9db4-7dd93a66c95e service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:31.531704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 699/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:40.786659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=35 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:41.529289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:46.916337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fcfd6-dc86-4503-94c2-af6ebc06fc1b None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:46.916880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fcfd6-dc86-4503-94c2-af6ebc06fc1b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:46.916880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fcfd6-dc86-4503-94c2-af6ebc06fc1b None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:46.918273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 711/1410] 10.222.0.73 () {54 vars in 918 bytes} [Wed Aug 24 05:39:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:46.923549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde70b-32bb-4a84-85bc-c768a874ed40 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:46.923999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde70b-32bb-4a84-85bc-c768a874ed40 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:46.924110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffde70b-32bb-4a84-85bc-c768a874ed40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:46.999300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-8ffde70b-32bb-4a84-85bc-c768a874ed40 None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:47.000489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8ffde70b-32bb-4a84-85bc-c768a874ed40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:47.106356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 700/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 05:39:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:47.122525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641cb4e-3a76-428c-9d06-13f3ae53c447 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:47.122805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641cb4e-3a76-428c-9d06-13f3ae53c447 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:47.123071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641cb4e-3a76-428c-9d06-13f3ae53c447 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:47.124424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 712/1412] 10.222.0.73 () {54 vars in 918 bytes} [Wed Aug 24 05:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:47.129845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e607d-2893-4b64-a9fb-6bfc0750d4f2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:47.130135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e607d-2893-4b64-a9fb-6bfc0750d4f2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:47.130404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e607d-2893-4b64-a9fb-6bfc0750d4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:47.202037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:47.205270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-4e0e607d-2893-4b64-a9fb-6bfc0750d4f2 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:47.207012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4e0e607d-2893-4b64-a9fb-6bfc0750d4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:47.320807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 701/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Wed Aug 24 05:39:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:49.705584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1f6ea-4b94-4363-87e9-42ccd0449b73 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:49.706149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1f6ea-4b94-4363-87e9-42ccd0449b73 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:49.706149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf1f6ea-4b94-4363-87e9-42ccd0449b73 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:49.707461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:39:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:49.712736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febb4f-f060-4ce9-8cd7-7b0fde4e8ef5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:49.713212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febb4f-f060-4ce9-8cd7-7b0fde4e8ef5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:49.713309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73febb4f-f060-4ce9-8cd7-7b0fde4e8ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:49.779428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-73febb4f-f060-4ce9-8cd7-7b0fde4e8ef5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:49.780544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-73febb4f-f060-4ce9-8cd7-7b0fde4e8ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:49.835698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:49.880335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:39:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:50.166083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1f6ea-4b94-4363-87e9-42ccd0449b73 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:50.167692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6cf1f6ea-4b94-4363-87e9-42ccd0449b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:50.310560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf1f6ea-4b94-4363-87e9-42ccd0449b73 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:50.382704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:50.422393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:50.423520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:50.423793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:50.424055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:50.425654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:50.518222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:50.519818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e7dccd-5aa1-4e23-b3bd-17fe26d0e400 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:50.538400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 714/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:51.434021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a20bf8-a665-4a23-a512-377fcac65f89 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:51.434589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a20bf8-a665-4a23-a512-377fcac65f89 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:51.434589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a20bf8-a665-4a23-a512-377fcac65f89 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:51.435925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:51.441463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816c0a8-148e-43e2-bf72-d104f83da66a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:51.441892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816c0a8-148e-43e2-bf72-d104f83da66a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:51.442045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c816c0a8-148e-43e2-bf72-d104f83da66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:51.520000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c816c0a8-148e-43e2-bf72-d104f83da66a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:51.521310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c816c0a8-148e-43e2-bf72-d104f83da66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:51.548221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:51.617639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:39:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:51.767482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a20bf8-a665-4a23-a512-377fcac65f89 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:51.769019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-48a20bf8-a665-4a23-a512-377fcac65f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:51.916213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a20bf8-a665-4a23-a512-377fcac65f89 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:51.955600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:52.028899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:52.029527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:52.029830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:52.030095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:52.031681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:52.119512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:52.121090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7e4a15-5d04-49db-be02-5c4821474d6a service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:52.138574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 704/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:53.049362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9f5e-9aec-42db-b986-e6923e9d1b11 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:53.049954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9f5e-9aec-42db-b986-e6923e9d1b11 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:53.049954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1e9f5e-9aec-42db-b986-e6923e9d1b11 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:53.051249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:53.056134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f81ba-b85a-4588-a885-7af1c7bda7f2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:53.056559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f81ba-b85a-4588-a885-7af1c7bda7f2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:53.056717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27f81ba-b85a-4588-a885-7af1c7bda7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:53.111792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:53.134741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a27f81ba-b85a-4588-a885-7af1c7bda7f2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:53.135945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a27f81ba-b85a-4588-a885-7af1c7bda7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:53.244241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:39:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:53.394859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9f5e-9aec-42db-b986-e6923e9d1b11 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:53.396386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aa1e9f5e-9aec-42db-b986-e6923e9d1b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:53.541514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1e9f5e-9aec-42db-b986-e6923e9d1b11 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:53.554113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:53.656339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:53.656900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:53.657187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:53.657461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:53.659102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:53.747661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:53.749251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f80144-2910-4de0-aef8-c899baabe8e8 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:53.767583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 717/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:54.633779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a40f46-fb82-4eed-8872-e927e71f2968 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:54.634340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a40f46-fb82-4eed-8872-e927e71f2968 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:54.634340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a40f46-fb82-4eed-8872-e927e71f2968 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:54.635688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:54.641190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7dbe-b90e-4f0c-a762-b973cd8d6c38 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:54.642172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7dbe-b90e-4f0c-a762-b973cd8d6c38 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:54.642367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7dbe-b90e-4f0c-a762-b973cd8d6c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:54.661715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:54.718323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-088a7dbe-b90e-4f0c-a762-b973cd8d6c38 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:54.719545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-088a7dbe-b90e-4f0c-a762-b973cd8d6c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:54.817005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:39:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:39:54.968026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a40f46-fb82-4eed-8872-e927e71f2968 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:54.969623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-83a40f46-fb82-4eed-8872-e927e71f2968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:55.079686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:55.116245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a40f46-fb82-4eed-8872-e927e71f2968 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:55.227409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:55.228193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:55.228477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:55.228751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:55.230393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:55.309081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:55.323407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:55.325035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a55e99-53aa-4049-a9d3-c4adedddf94c service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:55.343199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 707/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:56.205059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ebac6-c14f-498e-9c50-4cbf654d8e77 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:56.205650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ebac6-c14f-498e-9c50-4cbf654d8e77 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:56.205650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ebac6-c14f-498e-9c50-4cbf654d8e77 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:56.206969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:39:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:56.211918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cda10-74bd-42fc-bb15-3ba31eb99b65 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:56.212321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cda10-74bd-42fc-bb15-3ba31eb99b65 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:56.212468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846cda10-74bd-42fc-bb15-3ba31eb99b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:56.289482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-846cda10-74bd-42fc-bb15-3ba31eb99b65 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:56.290735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-846cda10-74bd-42fc-bb15-3ba31eb99b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:56.388143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:39:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:56.539513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ebac6-c14f-498e-9c50-4cbf654d8e77 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:56.540951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3e8ebac6-c14f-498e-9c50-4cbf654d8e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:56.624379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:56.686617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ebac6-c14f-498e-9c50-4cbf654d8e77 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:56.799784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:56.800416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:56.800698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:56.800969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:56.802729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:56.852478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:56.894132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:56.895709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74e350-469c-4189-a299-022a6a38b9a3 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:56.913602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 720/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:57.709874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7c1bf-b0a8-481d-800a-29cc41dfcd7b None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:57.710429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7c1bf-b0a8-481d-800a-29cc41dfcd7b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:57.710429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa7c1bf-b0a8-481d-800a-29cc41dfcd7b None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:57.711808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:39:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:57.716639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09bbc-2153-4893-a24b-8a712ce4be36 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:57.717078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09bbc-2153-4893-a24b-8a712ce4be36 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:57.717193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b09bbc-2153-4893-a24b-8a712ce4be36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:57.796165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-31b09bbc-2153-4893-a24b-8a712ce4be36 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:57.797390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-31b09bbc-2153-4893-a24b-8a712ce4be36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:57.893040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:39:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:58.046586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7c1bf-b0a8-481d-800a-29cc41dfcd7b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:58.048183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-faa7c1bf-b0a8-481d-800a-29cc41dfcd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:58.097850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:58.193732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa7c1bf-b0a8-481d-800a-29cc41dfcd7b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:58.305447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:58.306017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:58.306301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:58.306575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:58.308172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:58.315191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:58.398675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:58.400292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53548a2-3c71-42fb-a940-22c05a045a4a service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:58.418023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 710/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:39:59.213789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfcc1-dfef-47a1-8788-cce21fcfcda6 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:59.214334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfcc1-dfef-47a1-8788-cce21fcfcda6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:59.214334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0cfcc1-dfef-47a1-8788-cce21fcfcda6 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:59.215663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:39:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:39:59.221013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dabd79-2e12-42bc-b0e4-4fab2df28344 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:59.221410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dabd79-2e12-42bc-b0e4-4fab2df28344 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:59.221557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dabd79-2e12-42bc-b0e4-4fab2df28344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:59.298529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d9dabd79-2e12-42bc-b0e4-4fab2df28344 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:39:59.299724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d9dabd79-2e12-42bc-b0e4-4fab2df28344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:59.371754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:59.401140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:39:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:39:59.551466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cfcc1-dfef-47a1-8788-cce21fcfcda6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:39:59.552979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4f0cfcc1-dfef-47a1-8788-cce21fcfcda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:59.570381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:59.697318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0cfcc1-dfef-47a1-8788-cce21fcfcda6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:39:59.788266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:39:59.814688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:39:59.815348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:39:59.815649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:39:59.815921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:39:59.817536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:39:59.907103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:39:59.908683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7843dba5-23d4-4314-a6c4-63d16532b04d service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:39:59.926785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 723/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:39:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:00.737442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aab00-2f4c-418d-bc58-7e2d92015eb8 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:00.737973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aab00-2f4c-418d-bc58-7e2d92015eb8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:00.738076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663aab00-2f4c-418d-bc58-7e2d92015eb8 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:00.739360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:40:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:00.744305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264161-ea63-4d00-8561-143b56c27ceb None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:00.744702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264161-ea63-4d00-8561-143b56c27ceb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:00.744886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41264161-ea63-4d00-8561-143b56c27ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:00.822699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-41264161-ea63-4d00-8561-143b56c27ceb None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:00.823915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-41264161-ea63-4d00-8561-143b56c27ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:00.865220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:00.919748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:40:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:40:01.072123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aab00-2f4c-418d-bc58-7e2d92015eb8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:01.073764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-663aab00-2f4c-418d-bc58-7e2d92015eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:01.217029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663aab00-2f4c-418d-bc58-7e2d92015eb8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:01.271683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:01.330912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:01.331518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:01.331799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:01.332072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:01.333702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:01.422442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:01.424647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bccc9e3-a476-43a4-84c7-71fc89bb9aa8 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:01.442576 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 713/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:02.260047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fbaf9-0461-448f-95c5-626ce4ff1672 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:02.260607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fbaf9-0461-448f-95c5-626ce4ff1672 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:02.260607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fbaf9-0461-448f-95c5-626ce4ff1672 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:02.261985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:40:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:02.267263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29f982-d820-4259-8230-c9eb3320adad None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:02.267731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29f982-d820-4259-8230-c9eb3320adad None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:02.267838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de29f982-d820-4259-8230-c9eb3320adad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:02.342652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-de29f982-d820-4259-8230-c9eb3320adad None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:02.343848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-de29f982-d820-4259-8230-c9eb3320adad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:02.345860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:02.442442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:40:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:02.595758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fbaf9-0461-448f-95c5-626ce4ff1672 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:02.597322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-670fbaf9-0461-448f-95c5-626ce4ff1672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:02.743132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fbaf9-0461-448f-95c5-626ce4ff1672 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:02.770358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:02.861654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:02.862169 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:02.862450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:02.862723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:02.864346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:02.953732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:02.955313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2ebc9e-389c-4c60-94cb-2e34b79f11dc service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:02.973400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 726/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:04.806431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f7f93-d19b-4ace-aac6-e3b1e177f264 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:04.807013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f7f93-d19b-4ace-aac6-e3b1e177f264 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:04.807013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071f7f93-d19b-4ace-aac6-e3b1e177f264 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:04.808322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:04.814414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04872f6e-9a0e-442e-b2b4-a16150d73f77 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:04.814849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04872f6e-9a0e-442e-b2b4-a16150d73f77 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:04.815001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04872f6e-9a0e-442e-b2b4-a16150d73f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:04.856630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:04.883485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-04872f6e-9a0e-442e-b2b4-a16150d73f77 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:04.884700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-04872f6e-9a0e-442e-b2b4-a16150d73f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:04.979851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:05.270079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f7f93-d19b-4ace-aac6-e3b1e177f264 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:05.271935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-071f7f93-d19b-4ace-aac6-e3b1e177f264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:05.375773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071f7f93-d19b-4ace-aac6-e3b1e177f264 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:05.407619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:05.449066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:05.449684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:05.449965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:05.450238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:05.451835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:05.562076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:05.563478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c8235e-cb31-4cbc-be8f-9ee3057eca76 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:05.581718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 716/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:07.647527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84380f-7a1d-4374-b963-01c9790b7743 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:07.648080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84380f-7a1d-4374-b963-01c9790b7743 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:07.648080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84380f-7a1d-4374-b963-01c9790b7743 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:07.649446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:07.654679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2281-e711-4ce4-ac21-501135cd7e28 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:07.655080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2281-e711-4ce4-ac21-501135cd7e28 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:07.655223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcf2281-e711-4ce4-ac21-501135cd7e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:07.721741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2dcf2281-e711-4ce4-ac21-501135cd7e28 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:07.722955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2dcf2281-e711-4ce4-ac21-501135cd7e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:07.736442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:07.819782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:08.105717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84380f-7a1d-4374-b963-01c9790b7743 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:08.107244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8c84380f-7a1d-4374-b963-01c9790b7743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:08.212692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84380f-7a1d-4374-b963-01c9790b7743 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:08.226456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:08.302426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:08.303035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:08.303313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:08.303582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:08.305166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:08.412573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:08.413963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e41c09-e5a4-4f01-9706-d41d7ed2f6cf service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:08.432600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 729/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:10.655509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d8098-ca16-41ff-82f3-aa24d0446b60 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:10.656049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d8098-ca16-41ff-82f3-aa24d0446b60 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:10.656049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d8098-ca16-41ff-82f3-aa24d0446b60 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:10.657426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:10.662707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c2b93-8ec3-4810-b3d6-222d73dba3df None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:10.663108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c2b93-8ec3-4810-b3d6-222d73dba3df None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:10.663263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c2b93-8ec3-4810-b3d6-222d73dba3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:10.712103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:10.729938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c57c2b93-8ec3-4810-b3d6-222d73dba3df None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:10.731140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c57c2b93-8ec3-4810-b3d6-222d73dba3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:10.826323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:11.110749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d8098-ca16-41ff-82f3-aa24d0446b60 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:11.112291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d64d8098-ca16-41ff-82f3-aa24d0446b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:11.214692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d8098-ca16-41ff-82f3-aa24d0446b60 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:11.255692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:11.290582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:11.291190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:11.291470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:11.291741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:11.293334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:11.402151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:11.403505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c325728-61e6-46fb-8616-9a7db925bb7c service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:11.420970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 719/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:13.441458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f0f37-65db-468e-8539-7cce793a6715 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:13.442015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f0f37-65db-468e-8539-7cce793a6715 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:13.442130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f0f37-65db-468e-8539-7cce793a6715 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:13.443383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:13.448571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb22c0-2f2c-4c1e-a968-68c4ea288919 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:13.448973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb22c0-2f2c-4c1e-a968-68c4ea288919 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:13.449123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb22c0-2f2c-4c1e-a968-68c4ea288919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:13.515157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3bcb22c0-2f2c-4c1e-a968-68c4ea288919 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:13.516393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3bcb22c0-2f2c-4c1e-a968-68c4ea288919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:13.537251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:13.613447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:13.897238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f0f37-65db-468e-8539-7cce793a6715 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:13.898817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3a8f0f37-65db-468e-8539-7cce793a6715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:14.000994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f0f37-65db-468e-8539-7cce793a6715 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:14.021415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:14.076226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:14.076651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:14.076934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:14.077208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:14.079339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:14.186646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:14.187995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199b83c9-8cbb-42c3-8127-576fc9af6fc3 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:14.208479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 732/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:40:16.175722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215faa-dc1a-4e96-b853-cf35817c32c4 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:16.176281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215faa-dc1a-4e96-b853-cf35817c32c4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:16.176281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6215faa-dc1a-4e96-b853-cf35817c32c4 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:16.177627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:16.182943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15532-e73e-4916-8702-0f5011c7de28 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:16.183372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15532-e73e-4916-8702-0f5011c7de28 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:16.183514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15532-e73e-4916-8702-0f5011c7de28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:16.247096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:16.251903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3ec15532-e73e-4916-8702-0f5011c7de28 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:16.253312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ec15532-e73e-4916-8702-0f5011c7de28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:16.348583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:16.635843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215faa-dc1a-4e96-b853-cf35817c32c4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:16.637435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f6215faa-dc1a-4e96-b853-cf35817c32c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:16.744187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6215faa-dc1a-4e96-b853-cf35817c32c4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:16.793384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:16.819148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:16.819750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:16.820032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:16.820298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:16.821933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:16.932751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:16.934140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615e57e-163e-4869-9f1e-5e23bff4172e service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:16.952475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 722/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:18.951014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec8460-5dce-4219-82b1-c7db36d78c53 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:18.951616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec8460-5dce-4219-82b1-c7db36d78c53 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:18.951616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec8460-5dce-4219-82b1-c7db36d78c53 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:18.952899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:40:18.958236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a88b-23e1-49eb-b94d-7d67a21b720a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:18.958635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a88b-23e1-49eb-b94d-7d67a21b720a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:18.958792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b018a88b-23e1-49eb-b94d-7d67a21b720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:19.025584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b018a88b-23e1-49eb-b94d-7d67a21b720a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:19.026820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b018a88b-23e1-49eb-b94d-7d67a21b720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:19.056457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:19.125134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:19.412271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec8460-5dce-4219-82b1-c7db36d78c53 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:19.413844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e4ec8460-5dce-4219-82b1-c7db36d78c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:19.516388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec8460-5dce-4219-82b1-c7db36d78c53 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:19.544973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:19.590743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:19.591348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:19.591625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:19.591901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:19.593489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:19.700027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:19.701454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff87b6f-1410-4a41-8f3a-0622acdc2255 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:19.720029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 735/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:21.678280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d10435-24ed-49fc-b000-0e85d6800a02 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:21.678821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d10435-24ed-49fc-b000-0e85d6800a02 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:21.678821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d10435-24ed-49fc-b000-0e85d6800a02 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:21.680172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:21.685402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812d6ae-a66f-4683-b4d3-906915ddeac0 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:21.685877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812d6ae-a66f-4683-b4d3-906915ddeac0 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:21.686085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812d6ae-a66f-4683-b4d3-906915ddeac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:21.752063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3812d6ae-a66f-4683-b4d3-906915ddeac0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:21.753878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3812d6ae-a66f-4683-b4d3-906915ddeac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:21.762108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:21.851775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:22.138032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d10435-24ed-49fc-b000-0e85d6800a02 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:22.139582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-35d10435-24ed-49fc-b000-0e85d6800a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:22.242869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d10435-24ed-49fc-b000-0e85d6800a02 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:22.298971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:22.318975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:22.319586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:22.319861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:22.320144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:22.321741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:22.431401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:22.432763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd7d0c-95cc-4c5a-a526-98dfe2a68fdd service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:22.450981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 725/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:24.541022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522602a-c3af-4c2e-b60e-3709972f4365 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:24.541544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522602a-c3af-4c2e-b60e-3709972f4365 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:24.541544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522602a-c3af-4c2e-b60e-3709972f4365 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:24.543114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:24.548836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e511aa-cf30-4951-aef3-c03aaa5d1f8d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:24.549844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e511aa-cf30-4951-aef3-c03aaa5d1f8d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:24.549942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e511aa-cf30-4951-aef3-c03aaa5d1f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:24.616074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-43e511aa-cf30-4951-aef3-c03aaa5d1f8d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:24.617278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-43e511aa-cf30-4951-aef3-c03aaa5d1f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:24.653952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:24.713382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:24.997303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522602a-c3af-4c2e-b60e-3709972f4365 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:24.998889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c522602a-c3af-4c2e-b60e-3709972f4365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:25.101285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522602a-c3af-4c2e-b60e-3709972f4365 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:25.138622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:25.175481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:25.176088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:25.176368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:25.176640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:25.178280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:25.286952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:25.288420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e941238-b8eb-45aa-bb18-13579875091e service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:25.307151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 738/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:27.217821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b215d6-0774-42cb-9d7e-94783eb08d7c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:27.218418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b215d6-0774-42cb-9d7e-94783eb08d7c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:27.218418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b215d6-0774-42cb-9d7e-94783eb08d7c None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:27.219717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:27.225872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71790bb5-e04f-47c3-a784-c2ee16f897a8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:27.226289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71790bb5-e04f-47c3-a784-c2ee16f897a8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:27.226428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71790bb5-e04f-47c3-a784-c2ee16f897a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:27.292322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-71790bb5-e04f-47c3-a784-c2ee16f897a8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:27.293531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-71790bb5-e04f-47c3-a784-c2ee16f897a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:27.311146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:27.390630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:27.678239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b215d6-0774-42cb-9d7e-94783eb08d7c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:27.679871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c3b215d6-0774-42cb-9d7e-94783eb08d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:27.781807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b215d6-0774-42cb-9d7e-94783eb08d7c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:27.846798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:27.854588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:27.855194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:27.855468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:27.855734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:27.857308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:27.964123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:27.965586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573c8b74-e9b1-4fca-a5b3-7fa44cc34d5d service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:27.983428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 728/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:29.903776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658024fe-a2ca-4ec6-aabb-c752ce83a76d None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:29.904337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658024fe-a2ca-4ec6-aabb-c752ce83a76d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:29.904337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658024fe-a2ca-4ec6-aabb-c752ce83a76d None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:29.905687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:40:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:29.911290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca07e88-b963-4a54-adbe-e4144dcd29be None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:29.911684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca07e88-b963-4a54-adbe-e4144dcd29be None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:29.911843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca07e88-b963-4a54-adbe-e4144dcd29be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:29.978247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2ca07e88-b963-4a54-adbe-e4144dcd29be None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:29.979707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2ca07e88-b963-4a54-adbe-e4144dcd29be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:30.027352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:30.080777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:30.364501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658024fe-a2ca-4ec6-aabb-c752ce83a76d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:30.366111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-658024fe-a2ca-4ec6-aabb-c752ce83a76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:30.468454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658024fe-a2ca-4ec6-aabb-c752ce83a76d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:30.515825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:30.541719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:30.542316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:30.542590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:30.542899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:30.544819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:30.651359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:30.652718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfad530-ef9c-4762-9a87-a21e41dbe2c2 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:30.672047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 741/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:40.078043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=22 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:40.669584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=66 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:50.154668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54477f9a-0d78-4d86-b089-8e24374929e3 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:50.155305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54477f9a-0d78-4d86-b089-8e24374929e3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:50.155305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54477f9a-0d78-4d86-b089-8e24374929e3 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:50.156528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 730/1471] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:50.162299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbfcdc-543a-4855-8cbd-4e2eec229951 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:50.162699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbfcdc-543a-4855-8cbd-4e2eec229951 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:50.162871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fbfcdc-543a-4855-8cbd-4e2eec229951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:50.228324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b1fbfcdc-543a-4855-8cbd-4e2eec229951 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:50.229848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b1fbfcdc-543a-4855-8cbd-4e2eec229951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:50.327478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 742/1472] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:40:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:40:50.628005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514a04-9be5-4ffb-826d-863fcaaef37b None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:50.628295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514a04-9be5-4ffb-826d-863fcaaef37b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:50.628625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a514a04-9be5-4ffb-826d-863fcaaef37b None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:50.629975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 731/1473] 10.222.0.73 () {54 vars in 967 bytes} [Wed Aug 24 05:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:50.634823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450f27-3cd3-4f93-bffc-79550948ba89 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:50.635114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450f27-3cd3-4f93-bffc-79550948ba89 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:50.635391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75450f27-3cd3-4f93-bffc-79550948ba89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:50.712082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-75450f27-3cd3-4f93-bffc-79550948ba89 None None] MFA Rules not processed for user `960468d880f34ea2a267b6ada5fb30bd`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:50.713302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-75450f27-3cd3-4f93-bffc-79550948ba89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:50.734836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:50.789672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 743/1474] 10.222.0.73 () {58 vars in 1064 bytes} [Wed Aug 24 05:40:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:50.797262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a514a04-9be5-4ffb-826d-863fcaaef37b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:50.798793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5a514a04-9be5-4ffb-826d-863fcaaef37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:50.902082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a514a04-9be5-4ffb-826d-863fcaaef37b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:50.975172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:50.975783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:50.976091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:50.976366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:50.978039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:51.033550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:51.088845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:51.090324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea6c8c-6fb7-4f21-b03c-a77060d39335 service glance] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:51.108871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 732/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:53.088763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a422-a584-482e-b0f5-37f36c95c7fb None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:53.089323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a422-a584-482e-b0f5-37f36c95c7fb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:53.089323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141a422-a584-482e-b0f5-37f36c95c7fb None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:53.090682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 744/1476] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:40:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:53.095924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250bd3-807f-427c-950c-33dcf8ba47b7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:53.097030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250bd3-807f-427c-950c-33dcf8ba47b7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:53.097030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e250bd3-807f-427c-950c-33dcf8ba47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:53.163592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0e250bd3-807f-427c-950c-33dcf8ba47b7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:53.164812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0e250bd3-807f-427c-950c-33dcf8ba47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:53.262225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 733/1477] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:40:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:53.532500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a422-a584-482e-b0f5-37f36c95c7fb None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:53.534195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6141a422-a584-482e-b0f5-37f36c95c7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:53.659272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141a422-a584-482e-b0f5-37f36c95c7fb None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:53.707649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:53.752870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:53.753522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:53.753833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:53.754106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:53.754741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:53.756492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:40:53.756492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:40:53.756492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:40:53.757716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ce25f8-1d7c-4d96-aa82-aff55def0d09 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:53.776401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 745/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Wed Aug 24 05:40:53 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:40:55.721337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7c0c-b34c-445c-a6b1-a7fb1a32d387 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:55.722037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7c0c-b34c-445c-a6b1-a7fb1a32d387 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:55.722037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a7c0c-b34c-445c-a6b1-a7fb1a32d387 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:55.723628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 734/1479] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:55.728818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df7d6-8362-4867-a4cd-69754b73f9fe None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:55.729544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df7d6-8362-4867-a4cd-69754b73f9fe None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:55.729682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114df7d6-8362-4867-a4cd-69754b73f9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:55.794106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-114df7d6-8362-4867-a4cd-69754b73f9fe None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:55.795250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-114df7d6-8362-4867-a4cd-69754b73f9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:55.892115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 746/1480] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:40:56.187147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a7c0c-b34c-445c-a6b1-a7fb1a32d387 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:56.188763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-132a7c0c-b34c-445c-a6b1-a7fb1a32d387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:56.199047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:56.338259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a7c0c-b34c-445c-a6b1-a7fb1a32d387 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:56.418251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:56.452051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:56.452582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:56.452861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:56.453131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:56.454776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:56.544402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:56.546302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ae9c3-cbc1-4369-a9b0-47a6c3e44490 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:56.564720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 735/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:40:58.639112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b648a-f989-4470-ae1c-7c798d1e89e9 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:58.639713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b648a-f989-4470-ae1c-7c798d1e89e9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:58.639713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b648a-f989-4470-ae1c-7c798d1e89e9 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:58.640984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 747/1482] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:40:58.646878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5cc1-f391-4cef-a9b5-fe029d9ed46a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:58.647293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5cc1-f391-4cef-a9b5-fe029d9ed46a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:58.647399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab5cc1-f391-4cef-a9b5-fe029d9ed46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:58.711798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-11ab5cc1-f391-4cef-a9b5-fe029d9ed46a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:40:58.712983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-11ab5cc1-f391-4cef-a9b5-fe029d9ed46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:58.763020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:58.814704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 736/1483] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:40:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:40:59.105145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b648a-f989-4470-ae1c-7c798d1e89e9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:40:59.106744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-297b648a-f989-4470-ae1c-7c798d1e89e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:59.130393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:59.210140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b648a-f989-4470-ae1c-7c798d1e89e9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:40:59.284407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:40:59.285516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:40:59.285827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:40:59.286103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:40:59.287807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:40:59.362463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:40:59.397137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:40:59.398610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1375f019-dbbb-4dcb-9c9f-b2c17077d078 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:40:59.417267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 748/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:41:01.393678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb180-e67b-45cf-a023-23f8672eb666 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:01.394810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb180-e67b-45cf-a023-23f8672eb666 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:01.394810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661eb180-e67b-45cf-a023-23f8672eb666 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:01.396128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 737/1485] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:41:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:41:01.401390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc11b1-b29b-46ec-92cb-800867a35011 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:01.401851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc11b1-b29b-46ec-92cb-800867a35011 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:01.402097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc11b1-b29b-46ec-92cb-800867a35011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:01.467212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d6bc11b1-b29b-46ec-92cb-800867a35011 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:41:01.468403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d6bc11b1-b29b-46ec-92cb-800867a35011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:01.566494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 749/1486] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:41:01.857775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb180-e67b-45cf-a023-23f8672eb666 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:41:01.859365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-661eb180-e67b-45cf-a023-23f8672eb666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:01.963227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661eb180-e67b-45cf-a023-23f8672eb666 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:41:02.039040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:41:02.040904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:41:02.041596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:02.042051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:02.042374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:02.044343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:02.154566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:41:02.155855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc0069-904c-4d70-bf60-e771160fde84 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:41:02.174016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 738/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:41:04.275622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d799-03b3-41cc-a25c-cbcfc7599334 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:04.276067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d799-03b3-41cc-a25c-cbcfc7599334 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:04.276178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e403d799-03b3-41cc-a25c-cbcfc7599334 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:04.277494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 750/1488] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:41:04.282900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c344-c374-4b19-8010-60d9dcdb89e7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:04.283352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c344-c374-4b19-8010-60d9dcdb89e7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:04.283450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004c344-c374-4b19-8010-60d9dcdb89e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:04.349794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-c004c344-c374-4b19-8010-60d9dcdb89e7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:41:04.350990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c004c344-c374-4b19-8010-60d9dcdb89e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:04.418800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:41:04.466269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 739/1489] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:41:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:41:04.770360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403d799-03b3-41cc-a25c-cbcfc7599334 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:41:04.771915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e403d799-03b3-41cc-a25c-cbcfc7599334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:04.807345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:41:04.874587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e403d799-03b3-41cc-a25c-cbcfc7599334 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:41:04.946719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:41:04.947325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:04.947605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:04.947878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:04.949469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:05.028701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:41:05.058479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:41:05.059906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26d12c-51d5-4520-8228-e8db51a7d606 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:41:05.079236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 751/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:41:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:41:07.144754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fc0fd-9e78-467f-bb74-f1ec29a8cd3c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:07.145314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fc0fd-9e78-467f-bb74-f1ec29a8cd3c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:07.145314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fc0fd-9e78-467f-bb74-f1ec29a8cd3c None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:07.146670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 740/1491] 10.222.0.73 () {54 vars in 927 bytes} [Wed Aug 24 05:41:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:41:07.151889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d69e79-4825-4bc6-b75f-380cff9932b7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:07.152287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d69e79-4825-4bc6-b75f-380cff9932b7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:07.152445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d69e79-4825-4bc6-b75f-380cff9932b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:07.218140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-85d69e79-4825-4bc6-b75f-380cff9932b7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:41:07.219355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-85d69e79-4825-4bc6-b75f-380cff9932b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:07.317620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 752/1492] 10.222.0.73 () {58 vars in 1024 bytes} [Wed Aug 24 05:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:41:07.613006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fc0fd-9e78-467f-bb74-f1ec29a8cd3c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:41:07.614577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-af6fc0fd-9e78-467f-bb74-f1ec29a8cd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:07.838036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fc0fd-9e78-467f-bb74-f1ec29a8cd3c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:41:07.911028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:41:07.911632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:41:07.911910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:41:07.912182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:41:07.913809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:41:07.922289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:41:08.021698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:41:08.023128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd3099-0067-4b7b-ab36-d88b74582cea service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:41:08.041068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 741/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:41:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:41:17.315252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=82 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:41:18.038807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=50 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:18.848273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:18.848868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:18.848868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:18.925657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:18.926863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:19.021248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 753/1494] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:42:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:19.029311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:19.029833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:19.029929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:19.102807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:19.103914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:19.200269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 742/1495] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:42:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:19.205123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:19.206455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:19.218166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a984c-13ce-47d0-b05b-baf013c1c0b8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:19.280800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:19.314986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:19.315612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:19.315887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:19.316153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:19.316789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:42:19.317085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:19.318216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:19.318216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:42:19.318216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:42:19.319103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:19.330018 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 754/1496] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:42:19.338364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:19.347929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48dfb5a-92d1-4163-94c8-45180eba97b5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:19.409752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:19.443385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:19.443987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:19.444265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:19.444533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:19.445159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:19.446928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:19.446928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:42:19.446928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:42:19.448153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:19.504400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 743/1497] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:42:19 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:42:19.508556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:19.510231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:19.543615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823a22b-1f51-4a6c-9d87-18e44b11d8d3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:19.636864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:19.637465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:19.637771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:19.638031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:19.638651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:19.639645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:19.639645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:42:19.639645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:42:19.640850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:19.682706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 755/1498] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:42:19 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:42:19.686533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:19.688051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:19.749964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc047cbf-c87e-407b-a8a5-3d9b427d4c15 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:19.836351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:19.848200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:19.848805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:19.849083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:19.849353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:19.850046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:19.851098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:19.851098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:42:19.851098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:42:19.851982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:19.864488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 744/1499] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:42:19.868545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:19.870120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:19.881287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93de94-982c-4f9b-9572-2782b761b303 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:19.906565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:19.979953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:19.980471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:19.980748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:19.981277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:19.981947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:19.982948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:19.982948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:42:19.982948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:42:19.983830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb224c7-95d6-4c52-81c6-112c4997b797 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:19.995935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 756/1500] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:42:20.000189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:20.001749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:20.035195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a82979c-d7d8-48f9-b4fa-638580b20247 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:20.129073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:20.129762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.130046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.130321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] PATH_INFO: `/v3/projects/c087320c99c44eac9b7c39d765620711/users/a0d18cd61c6547599a5887fea6202021/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.151678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c087320c99c44eac9b7c39d765620711, user_id=a0d18cd61c6547599a5887fea6202021, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:20.152705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:20.152705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:42:20.152705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:42:20.155343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b0d7e1-5165-4a2c-82a5-c414f407c836 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:20.180003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 745/1501] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:42:19 2022] PUT /identity/v3/projects/c087320c99c44eac9b7c39d765620711/users/a0d18cd61c6547599a5887fea6202021/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:42:20.192186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84067c-d28d-4289-adb5-debc46b65004 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.192471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84067c-d28d-4289-adb5-debc46b65004 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.192743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84067c-d28d-4289-adb5-debc46b65004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.250031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cf84067c-d28d-4289-adb5-debc46b65004 None None] MFA Rules not processed for user `a0d18cd61c6547599a5887fea6202021`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:20.251151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cf84067c-d28d-4289-adb5-debc46b65004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:20.327799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 757/1502] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:42:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:20.340897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba07f-2d3a-4b22-9fdf-e05bf939a3a6 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.341182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba07f-2d3a-4b22-9fdf-e05bf939a3a6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.341450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baba07f-2d3a-4b22-9fdf-e05bf939a3a6 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.342876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 746/1503] 10.222.0.73 () {54 vars in 964 bytes} [Wed Aug 24 05:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:20.348020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909c07a-37dc-4f93-9f47-de3fdce7be14 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.348315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909c07a-37dc-4f93-9f47-de3fdce7be14 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.348614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909c07a-37dc-4f93-9f47-de3fdce7be14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.380366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:20.423819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a909c07a-37dc-4f93-9f47-de3fdce7be14 None None] MFA Rules not processed for user `1c7f12a5dc6446eaa94a96dbb44ccdaf`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:20.425668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a909c07a-37dc-4f93-9f47-de3fdce7be14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:20.501243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 758/1504] 10.222.0.73 () {58 vars in 1061 bytes} [Wed Aug 24 05:42:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:20.508974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baba07f-2d3a-4b22-9fdf-e05bf939a3a6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:20.510522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9baba07f-2d3a-4b22-9fdf-e05bf939a3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:20.561328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:20.619265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baba07f-2d3a-4b22-9fdf-e05bf939a3a6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:20.691198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:20.691794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.692072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.692340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.693961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:20.791284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:20.792631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:20.796370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:20.816493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 747/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:20.861687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3a91-16ef-465a-b1a4-467a0a9f3e26 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:20.861976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3a91-16ef-465a-b1a4-467a0a9f3e26 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:20.862253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd3a91-16ef-465a-b1a4-467a0a9f3e26 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:20.863504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 759/1506] 10.222.0.73 () {52 vars in 830 bytes} [Wed Aug 24 05:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:42:20.867014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:20.868223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:20.966647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4212d4aa-675b-451b-b1da-c85306526032 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:21.057422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:21.058071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:21.058263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:21.058529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] PATH_INFO: `/v3/users/a0d18cd61c6547599a5887fea6202021` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:21.071016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0d18cd61c6547599a5887fea6202021)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:21.072033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:21.072033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:42:21.072033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:42:21.073602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:21.084568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:21.152323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0d18cd61c6547599a5887fea6202021'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:21.152766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0d18cd61c6547599a5887fea6202021'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:21.169543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0d18cd61c6547599a5887fea6202021'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:21.170125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0d18cd61c6547599a5887fea6202021'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:21.179324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0d18cd61c6547599a5887fea6202021'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:21.209582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 748/1507] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:42:20 2022] DELETE /identity/v3/users/a0d18cd61c6547599a5887fea6202021 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:42:21.213948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:21.214455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:21.214555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:21.273267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:42:21.274526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:21.348756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:21.362082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 760/1508] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:42:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:42:21.372209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:21.373426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:21.481301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151a8681-a831-4d2b-9085-b5572c9f79b3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:21.538850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:21.559660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:21.560281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:21.560559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:21.560830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:21.562552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:21.672195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:21.673582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791269b4-3208-4965-b6dd-393b5cb5d3a8 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:21.692501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 749/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:42:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:42:22.038180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:42:22.039392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:42:22.051485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8bae94-384d-4a5b-b921-db1b8ce2ef6f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:42:22.155496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:42:22.156171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:42:22.156454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:42:22.156726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] PATH_INFO: `/v3/projects/c087320c99c44eac9b7c39d765620711` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:42:22.166799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c087320c99c44eac9b7c39d765620711)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:42:22.167861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:42:22.167861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:42:22.167861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:42:22.169097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:42:22.226075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c087320c99c44eac9b7c39d765620711', 'request_id': 'req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:22.232479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c087320c99c44eac9b7c39d765620711', 'request_id': 'req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:22.241915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c087320c99c44eac9b7c39d765620711', 'request_id': 'req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:22.272196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] The token cache is being invalidate because project c087320c99c44eac9b7c39d765620711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:42:22.272492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:22.272930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6b8f0715-997d-4eb9-8716-b3cf2f9081f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:42:22.274435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 761/1510] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:42:22 2022] DELETE /identity/v3/projects/c087320c99c44eac9b7c39d765620711 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:42:31.689665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=62 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:42:32.270153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:43:52.377655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417eb47-5d19-4f35-966c-6d062c81d25b None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:52.378241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417eb47-5d19-4f35-966c-6d062c81d25b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:52.378241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4417eb47-5d19-4f35-966c-6d062c81d25b None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:52.379591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:43:52.384463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9ce41-8357-454a-aa11-74c8a5f1cb05 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:52.384872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9ce41-8357-454a-aa11-74c8a5f1cb05 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:52.385008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9ce41-8357-454a-aa11-74c8a5f1cb05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:52.467774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-0ca9ce41-8357-454a-aa11-74c8a5f1cb05 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:43:52.468981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0ca9ce41-8357-454a-aa11-74c8a5f1cb05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:43:52.574372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:43:52.727133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417eb47-5d19-4f35-966c-6d062c81d25b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:43:52.729267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4417eb47-5d19-4f35-966c-6d062c81d25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:43:52.846227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4417eb47-5d19-4f35-966c-6d062c81d25b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:43:52.851616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847709b-f47f-48b7-a239-a569e696bcca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:43:52.852221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847709b-f47f-48b7-a239-a569e696bcca service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:52.852503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847709b-f47f-48b7-a239-a569e696bcca service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:52.852778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847709b-f47f-48b7-a239-a569e696bcca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:52.861618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847709b-f47f-48b7-a239-a569e696bcca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:43:52.863209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847709b-f47f-48b7-a239-a569e696bcca service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:43:52.872548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 751/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:43:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:43:53.698490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca44c5-956c-4ccc-8fbc-1daa3e480a6b None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:53.699010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca44c5-956c-4ccc-8fbc-1daa3e480a6b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:53.699010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca44c5-956c-4ccc-8fbc-1daa3e480a6b None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:53.700380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:43:53.705854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94b2ec-1e81-48f7-9659-c2c9b04227a7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:53.706297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94b2ec-1e81-48f7-9659-c2c9b04227a7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:53.706441 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a94b2ec-1e81-48f7-9659-c2c9b04227a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:53.727451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6a94b2ec-1e81-48f7-9659-c2c9b04227a7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:43:53.728654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6a94b2ec-1e81-48f7-9659-c2c9b04227a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:43:53.736878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:43:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:43:53.826978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e523-9145-4b4d-9d70-790571e7994b None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:53.828242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e523-9145-4b4d-9d70-790571e7994b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:53.828520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135e523-9145-4b4d-9d70-790571e7994b None None] PATH_INFO: `/v3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:53.830479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 764/1516] 10.222.0.57 () {54 vars in 925 bytes} [Wed Aug 24 05:43:53 2022] GET /identity/v3 => generated 255 bytes in 5 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:43:53.837124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255498b7-5dc0-4ff5-9dc6-451791ba8f7d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:53.837411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255498b7-5dc0-4ff5-9dc6-451791ba8f7d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:53.837745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255498b7-5dc0-4ff5-9dc6-451791ba8f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:53.898480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-255498b7-5dc0-4ff5-9dc6-451791ba8f7d None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:43:53.899608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-255498b7-5dc0-4ff5-9dc6-451791ba8f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:43:53.912010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e523-9145-4b4d-9d70-790571e7994b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:43:53.926455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135e523-9145-4b4d-9d70-790571e7994b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:43:53.932254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfffec-7315-4d44-8b58-5a968fe2d0a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:43:53.932868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfffec-7315-4d44-8b58-5a968fe2d0a2 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:43:53.933145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfffec-7315-4d44-8b58-5a968fe2d0a2 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:43:53.933410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfffec-7315-4d44-8b58-5a968fe2d0a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:43:53.942897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfffec-7315-4d44-8b58-5a968fe2d0a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:43:53.944482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfffec-7315-4d44-8b58-5a968fe2d0a2 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:43:53.954892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 765/1517] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:43:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:43:53.956004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:43:53.974899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 753/1518] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 24 05:43:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:03.133405 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a16d2-477d-4f26-8968-b603953e602e None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:03.134004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a16d2-477d-4f26-8968-b603953e602e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:03.134004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923a16d2-477d-4f26-8968-b603953e602e None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:03.135324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 766/1519] 10.222.0.73 () {54 vars in 961 bytes} [Wed Aug 24 05:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:03.140287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e58fb-1e19-48a5-95c7-d84bcd5e0ce9 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:03.140774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e58fb-1e19-48a5-95c7-d84bcd5e0ce9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:03.140893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e58fb-1e19-48a5-95c7-d84bcd5e0ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:03.161188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2f2e58fb-1e19-48a5-95c7-d84bcd5e0ce9 None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:03.162417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2f2e58fb-1e19-48a5-95c7-d84bcd5e0ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:03.170952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 754/1520] 10.222.0.73 () {58 vars in 1058 bytes} [Wed Aug 24 05:44:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:03.178997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a16d2-477d-4f26-8968-b603953e602e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:03.192570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923a16d2-477d-4f26-8968-b603953e602e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:03.197500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02506595-cb33-41dc-b830-6fb1ee5e916a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:03.198133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02506595-cb33-41dc-b830-6fb1ee5e916a service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:03.198533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02506595-cb33-41dc-b830-6fb1ee5e916a service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:03.198811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02506595-cb33-41dc-b830-6fb1ee5e916a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:03.207220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02506595-cb33-41dc-b830-6fb1ee5e916a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:03.208792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02506595-cb33-41dc-b830-6fb1ee5e916a service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:03.218915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 767/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:05.166072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17539e4-1b14-4f78-b08b-c18b445e0479 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:05.166639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17539e4-1b14-4f78-b08b-c18b445e0479 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:05.166639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17539e4-1b14-4f78-b08b-c18b445e0479 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:05.168054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:05.173348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a8ff-78d9-4f82-adfe-829b7f799b8c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:05.173855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a8ff-78d9-4f82-adfe-829b7f799b8c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:05.173962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8351a8ff-78d9-4f82-adfe-829b7f799b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:05.193465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8351a8ff-78d9-4f82-adfe-829b7f799b8c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:05.194691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8351a8ff-78d9-4f82-adfe-829b7f799b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:05.202346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:05.354880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17539e4-1b14-4f78-b08b-c18b445e0479 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:05.369621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17539e4-1b14-4f78-b08b-c18b445e0479 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:05.375038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d39f68-c31c-4901-a055-65d3e51bbcb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:05.375630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39f68-c31c-4901-a055-65d3e51bbcb6 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:05.376429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39f68-c31c-4901-a055-65d3e51bbcb6 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:05.376700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d39f68-c31c-4901-a055-65d3e51bbcb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:05.384781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d39f68-c31c-4901-a055-65d3e51bbcb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:05.386389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d39f68-c31c-4901-a055-65d3e51bbcb6 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:05.396416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 756/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:06.204786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a6b79-ae3a-412e-a6b9-0d94c66b2889 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:06.205351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a6b79-ae3a-412e-a6b9-0d94c66b2889 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:06.205351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a6b79-ae3a-412e-a6b9-0d94c66b2889 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:06.206681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 05:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:06.211382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60027a7c-0a52-44b5-b0cc-638ce0756193 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:06.211818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60027a7c-0a52-44b5-b0cc-638ce0756193 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:06.211922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60027a7c-0a52-44b5-b0cc-638ce0756193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:06.231622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-60027a7c-0a52-44b5-b0cc-638ce0756193 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:06.233072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-60027a7c-0a52-44b5-b0cc-638ce0756193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:06.240890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 05:44:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:06.391346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a6b79-ae3a-412e-a6b9-0d94c66b2889 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:06.406350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a6b79-ae3a-412e-a6b9-0d94c66b2889 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:06.412561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bc8144-0e6e-425d-9d1e-7c3584a85906 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:06.413163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc8144-0e6e-425d-9d1e-7c3584a85906 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:06.413444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc8144-0e6e-425d-9d1e-7c3584a85906 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:06.413749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bc8144-0e6e-425d-9d1e-7c3584a85906 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:06.422231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc8144-0e6e-425d-9d1e-7c3584a85906 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:06.423855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bc8144-0e6e-425d-9d1e-7c3584a85906 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:06.433272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 770/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:10.241854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbe2c-d3e1-4194-92b0-5e9f924b961b None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:10.242413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbe2c-d3e1-4194-92b0-5e9f924b961b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:10.242413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fbe2c-d3e1-4194-92b0-5e9f924b961b None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:10.243724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:10.249466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864090a-dd71-46ac-8c1c-6881b2459041 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:10.249968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864090a-dd71-46ac-8c1c-6881b2459041 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:10.250072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864090a-dd71-46ac-8c1c-6881b2459041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:10.269775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2864090a-dd71-46ac-8c1c-6881b2459041 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:10.271311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2864090a-dd71-46ac-8c1c-6881b2459041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:10.278884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:10.566831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fbe2c-d3e1-4194-92b0-5e9f924b961b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:10.568546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-259fbe2c-d3e1-4194-92b0-5e9f924b961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:10.645744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fbe2c-d3e1-4194-92b0-5e9f924b961b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:10.649332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260bbf6c-b255-42a1-9737-61f581d6912b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:10.649961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bbf6c-b255-42a1-9737-61f581d6912b service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:10.650247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bbf6c-b255-42a1-9737-61f581d6912b service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:10.650523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260bbf6c-b255-42a1-9737-61f581d6912b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:10.659855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bbf6c-b255-42a1-9737-61f581d6912b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:10.661283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260bbf6c-b255-42a1-9737-61f581d6912b service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:10.670638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 759/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:12.561557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdb8c6-0d8e-41d6-97e8-507404a99467 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:12.562108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdb8c6-0d8e-41d6-97e8-507404a99467 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:12.562108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fdb8c6-0d8e-41d6-97e8-507404a99467 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:12.564176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:44:12.569789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6a068-4543-4893-a7c4-be6ed45e6151 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:12.570274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6a068-4543-4893-a7c4-be6ed45e6151 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:12.570373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6a068-4543-4893-a7c4-be6ed45e6151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:12.590159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-23b6a068-4543-4893-a7c4-be6ed45e6151 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:12.591346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-23b6a068-4543-4893-a7c4-be6ed45e6151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:12.599173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:12.882813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdb8c6-0d8e-41d6-97e8-507404a99467 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:12.894197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fdb8c6-0d8e-41d6-97e8-507404a99467 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:12.897720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1eee0a-d42c-47bd-bfd5-94c5f0d6103b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:12.898318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eee0a-d42c-47bd-bfd5-94c5f0d6103b service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:12.898596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eee0a-d42c-47bd-bfd5-94c5f0d6103b service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:12.898868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1eee0a-d42c-47bd-bfd5-94c5f0d6103b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:12.907716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1eee0a-d42c-47bd-bfd5-94c5f0d6103b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:12.909132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1eee0a-d42c-47bd-bfd5-94c5f0d6103b service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:12.918465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 773/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:16.588016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37b769-0ab7-40c5-803b-ec47ef82473c None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:16.588546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37b769-0ab7-40c5-803b-ec47ef82473c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:16.588688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d37b769-0ab7-40c5-803b-ec47ef82473c None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:16.590155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:16.595955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00c2e8-74d0-42ab-a9f1-b6baa09b38f7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:16.596368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00c2e8-74d0-42ab-a9f1-b6baa09b38f7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:16.596476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00c2e8-74d0-42ab-a9f1-b6baa09b38f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:16.604044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:16.617933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3c00c2e8-74d0-42ab-a9f1-b6baa09b38f7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:16.619138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3c00c2e8-74d0-42ab-a9f1-b6baa09b38f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:16.627339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:17.180806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37b769-0ab7-40c5-803b-ec47ef82473c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:17.182495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5d37b769-0ab7-40c5-803b-ec47ef82473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:17.259689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d37b769-0ab7-40c5-803b-ec47ef82473c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:17.263493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb3e5d-59eb-49c7-8a19-7c4b9466a34d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:17.264103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb3e5d-59eb-49c7-8a19-7c4b9466a34d service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:17.264385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb3e5d-59eb-49c7-8a19-7c4b9466a34d service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:17.264658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb3e5d-59eb-49c7-8a19-7c4b9466a34d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:17.273832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb3e5d-59eb-49c7-8a19-7c4b9466a34d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:17.275271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bb3e5d-59eb-49c7-8a19-7c4b9466a34d service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:17.285424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 762/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:19.188212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407492d0-2a94-4dea-9100-728044b40f1a None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.188796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407492d0-2a94-4dea-9100-728044b40f1a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.188796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407492d0-2a94-4dea-9100-728044b40f1a None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.190118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 05:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:44:19.195264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9048e-b98e-4fe9-b9cd-36b2b1bfe3e9 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.195703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9048e-b98e-4fe9-b9cd-36b2b1bfe3e9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.195864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9048e-b98e-4fe9-b9cd-36b2b1bfe3e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.215404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-adc9048e-b98e-4fe9-b9cd-36b2b1bfe3e9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:44:19.216617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-adc9048e-b98e-4fe9-b9cd-36b2b1bfe3e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:19.224927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 05:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:44:19.772080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407492d0-2a94-4dea-9100-728044b40f1a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:44:19.773903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-407492d0-2a94-4dea-9100-728044b40f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:44:19.787005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407492d0-2a94-4dea-9100-728044b40f1a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:44:19.790527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe6e1d-bdc8-43cc-8215-7af5350c5198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:44:19.791121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe6e1d-bdc8-43cc-8215-7af5350c5198 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:44:19.791401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe6e1d-bdc8-43cc-8215-7af5350c5198 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:44:19.791701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe6e1d-bdc8-43cc-8215-7af5350c5198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:44:19.800973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe6e1d-bdc8-43cc-8215-7af5350c5198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:44:19.802427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe6e1d-bdc8-43cc-8215-7af5350c5198 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:44:19.812797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 776/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:44:29.210757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:44:29.807547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=8 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:43.419936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.420712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.420712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.447597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.448868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.465814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 764/1540] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:43.474490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c65ca3-bab7-42ce-a3f8-05feaa3dc7b4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.475476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c65ca3-bab7-42ce-a3f8-05feaa3dc7b4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.476009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c65ca3-bab7-42ce-a3f8-05feaa3dc7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.511116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-90c65ca3-bab7-42ce-a3f8-05feaa3dc7b4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.512700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-90c65ca3-bab7-42ce-a3f8-05feaa3dc7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.525781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 777/1541] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:43.531356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:43.547851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50595e1d-ce9e-41ee-b0ac-1f3a50963b2a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:43.558202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:43.558832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.559115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.559390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.560049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:43.560353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:43.561413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:45:43.561413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:45:43.561413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:45:43.562342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:43.570691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36239cb9-d46b-49df-8dbc-7a86080218a7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.571337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36239cb9-d46b-49df-8dbc-7a86080218a7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.571827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36239cb9-d46b-49df-8dbc-7a86080218a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.583966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 765/1542] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:43.592282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:43.610436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-36239cb9-d46b-49df-8dbc-7a86080218a7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.611795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdffbda-6ce3-40f6-9f6a-a029e3fd4fe0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:43.612275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-36239cb9-d46b-49df-8dbc-7a86080218a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.618864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:43.619480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.619763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.620038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.620662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:43.622631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:43.636103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 778/1543] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:43.645425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb16f3-9cf5-44e9-bb15-0037d880c7b0 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.646585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb16f3-9cf5-44e9-bb15-0037d880c7b0 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.647105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fb16f3-9cf5-44e9-bb15-0037d880c7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.680988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-70fb16f3-9cf5-44e9-bb15-0037d880c7b0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.682882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-70fb16f3-9cf5-44e9-bb15-0037d880c7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.700579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 779/1544] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:43.701918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.702206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.702477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.709667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 766/1545] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/projects => generated 363 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:45:43.711383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:43.738602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.739818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.836391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc02f1-1821-4715-a288-31b67d7b7699 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:43.844593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:43.845473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.846029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.846525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.847411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:43.847962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:43.849964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c96724-a814-4c12-bfa4-f5643b15e138 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:43.866770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 780/1546] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:43.870287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:43.879828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 767/1547] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:43.884210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8016f-bb38-4def-96f9-c93fb8850114 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.884210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8016f-bb38-4def-96f9-c93fb8850114 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.884210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8016f-bb38-4def-96f9-c93fb8850114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.910686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3d419-08cf-4f56-b215-51ae00ff1f0a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:43.917443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-83c8016f-bb38-4def-96f9-c93fb8850114 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:43.919966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-83c8016f-bb38-4def-96f9-c93fb8850114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:43.925777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffebc4f7-20fc-4557-ba3f-b5eab7ac76be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:43.928115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc4f7-20fc-4557-ba3f-b5eab7ac76be admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.928633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc4f7-20fc-4557-ba3f-b5eab7ac76be admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.929466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffebc4f7-20fc-4557-ba3f-b5eab7ac76be admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.931655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebc4f7-20fc-4557-ba3f-b5eab7ac76be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:43.935044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffebc4f7-20fc-4557-ba3f-b5eab7ac76be admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:43.945413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 768/1548] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:43.947164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.947658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.948245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.992740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 781/1549] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:43.994919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0295-7622-4324-8b3c-cabf81d73330 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:43.995419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0295-7622-4324-8b3c-cabf81d73330 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:43.996026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb0295-7622-4324-8b3c-cabf81d73330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:43.999449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.003358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.030938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 769/1550] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.032555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.045278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62f2309-a3cd-4c3b-9973-e86f85cc7e73 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.051231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-65eb0295-7622-4324-8b3c-cabf81d73330 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.051231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-65eb0295-7622-4324-8b3c-cabf81d73330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.070752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578fb2a-58f1-45be-bdaa-dfc809fe64e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.072142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fb2a-58f1-45be-bdaa-dfc809fe64e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.072437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fb2a-58f1-45be-bdaa-dfc809fe64e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.072715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fb2a-58f1-45be-bdaa-dfc809fe64e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.073619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578fb2a-58f1-45be-bdaa-dfc809fe64e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.073763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 782/1551] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.074706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.074991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.075268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.075993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578fb2a-58f1-45be-bdaa-dfc809fe64e7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.096255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.097384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.108192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 783/1552] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.109364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.118469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 770/1553] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.120356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305b2e-9dc5-4146-b184-f741e5c240ac None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.120643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305b2e-9dc5-4146-b184-f741e5c240ac None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.120917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305b2e-9dc5-4146-b184-f741e5c240ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.123691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38c2a6-c4b9-420f-8142-349cb447f829 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.140015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-cd305b2e-9dc5-4146-b184-f741e5c240ac None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.141226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cd305b2e-9dc5-4146-b184-f741e5c240ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.182493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1642131-596a-4976-80fd-98db0881d4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.183134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1642131-596a-4976-80fd-98db0881d4ab admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.183415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1642131-596a-4976-80fd-98db0881d4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.183685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1642131-596a-4976-80fd-98db0881d4ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.184328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1642131-596a-4976-80fd-98db0881d4ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.185993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1642131-596a-4976-80fd-98db0881d4ab admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.195882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 784/1554] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.197332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.197794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.198210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.203528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 771/1555] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.204877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.205279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.205723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.219247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.220467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.225315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.226526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.228861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 785/1556] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.229688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.235852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 772/1557] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.236705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.241334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.242269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb06c320-ac7b-452d-9f41-71f28204731d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.248199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.248796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d4fed-6ea5-4da1-8493-d07e30622c94 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.248978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.249261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.249535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.250222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.250524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.252137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa09a9-afe0-4980-bb03-f1fe8e6154de admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.253977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.254588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.254870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.255145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.255796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.256103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.257739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.267996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 786/1558] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.269539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.269539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.269539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.276823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 773/1559] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.277533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.287995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5876ba-3473-49e1-893b-dbef6ed831a6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.293254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.294014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.294299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.294578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.294812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.295201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.296670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.296794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:45:44.296794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:45:44.296794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:45:44.298005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.304570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 787/1560] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.305391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.316083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc8ba3-1d18-4186-a43c-5f0d1d528c1f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.320768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f18503-d9f0-402e-aba8-b714738901f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.321404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18503-d9f0-402e-aba8-b714738901f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.321532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 774/1561] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.321793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18503-d9f0-402e-aba8-b714738901f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.322000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f18503-d9f0-402e-aba8-b714738901f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.322600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.325553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f18503-d9f0-402e-aba8-b714738901f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.327232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f18503-d9f0-402e-aba8-b714738901f3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.332373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f1da9-4e34-4ca3-801f-caf2f8e9610b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.337080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.337703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.337987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.338122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 788/1562] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.338351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.338900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.339204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.340808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e8832-2f21-460c-ad6e-5ef6b5c22924 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.342016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.343127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.343412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.354079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 775/1563] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.359785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.360199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.360482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.363441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.364655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.372579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 789/1564] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.373333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.378525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.382463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.383681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.384870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3925c92-7172-4591-ab0f-678201b2dbe1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.390640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.391235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.391517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.391795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.396128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 776/1565] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.396275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.396387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.396493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:45:44.396493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:45:44.396493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:45:44.396493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecff4e7-cb82-48c3-b1ae-875521165b14 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.402642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477ddc7-b866-4896-8471-fe25c741ca81 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.408200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.408802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.409080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.409353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.410008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.411921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.423346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 790/1566] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.424602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.424885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.425158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.445601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 777/1567] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.446154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.448763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.450016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.508260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e080d5-c871-4e85-815a-3a1249bae358 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.508976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 791/1568] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.509853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.516476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.517082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.517364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.517669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.518317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.519236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e90eda-ea34-4b64-9503-a2a08adedd29 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.519990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.523920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.524563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.524842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.525109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] PATH_INFO: `/v3/projects/d12cf60eb26941e6a9b1dcfe56b3fa9e/users/fd2ee5dfc7a54e39b940f22da1ae9d3d/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.530178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 778/1569] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.530946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.539583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12cf60eb26941e6a9b1dcfe56b3fa9e, user_id=fd2ee5dfc7a54e39b940f22da1ae9d3d, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.540387 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89da426f-895d-4797-8717-4d119ef1d62a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.540623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:45:44.540623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:45:44.540623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:45:44.542649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.545028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.546173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.546454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.546745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.547379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.549861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.556045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 792/1570] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:44 2022] PUT /identity/v3/projects/d12cf60eb26941e6a9b1dcfe56b3fa9e/users/fd2ee5dfc7a54e39b940f22da1ae9d3d/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:44.556855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.578874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 779/1571] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.580074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.626526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009009d-efce-4226-878f-7e3a8d390fce admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.644654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b5df-b310-4b37-af7e-f1358845c609 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.649285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.649929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.650208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.650481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.651128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.651453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.653059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.660388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.669740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 780/1572] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.671003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.682308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abede27-c6f4-4a2e-806f-f29131ab04e0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.686688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.686888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.687303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.687468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.687578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.687743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.687840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.688035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.688472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.688644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.688759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.690388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.690585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.697000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:44.703520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 793/1573] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.704603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.713856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 781/1574] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.715324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d7f5d-e8ef-4b93-9a49-e03ce938d4ae admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.718302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.721316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc64c3-984e-44f8-995d-430621c5ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.721942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc64c3-984e-44f8-995d-430621c5ec3a admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.722220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc64c3-984e-44f8-995d-430621c5ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.722499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc64c3-984e-44f8-995d-430621c5ec3a admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.723109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc64c3-984e-44f8-995d-430621c5ec3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.725011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc64c3-984e-44f8-995d-430621c5ec3a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.727756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3babf15-b0ae-48f1-87c3-b700917a30b1 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.732320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.733251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.733459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.733766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.734400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:44.734703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.736277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.755848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 782/1575] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:44.756049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 794/1576] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/users => generated 286 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.756170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.756282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.756282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.756282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.761546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bf331-1d55-41ba-9249-78f0e2f2a90a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.766645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.767266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.767544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.767815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.768444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.770071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.779861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 783/1577] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.781218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.781781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] MFA Rules not processed for user `fd2ee5dfc7a54e39b940f22da1ae9d3d`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:44.782982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:44.791400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3240283-41b9-4995-aef4-7823cbf1b939 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.795978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.796562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.796840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.797114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.797759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.799661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.820696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 784/1578] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.822010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.833226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606f129-3ed0-4800-9822-135a7e55beb3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.837927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.838513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.838793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.839066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.839680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.841604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.850173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 795/1579] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:44.851181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.862302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81185519-fbfe-4427-b1b1-fa26b14c6312 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.867181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.867784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.868070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.868339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.868965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.874163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 785/1580] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.874337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.874433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.899125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 796/1581] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:44.900055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.934802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf022c-90f0-49a7-a0cb-7ee53ec4544e admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.964593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafa7ca-6add-4739-8480-387fe48aba92 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:44.969218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:44.969845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:44.970123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:44.970787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:44.971437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:44.973014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:44.983454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 797/1582] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:44.984260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:44.994826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64241-a2f0-46e8-b74b-0c0870ba1c38 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.000126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.000733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.001020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.001284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.001938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.002928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.003817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.003942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.004237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.004637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.005666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.008189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.021778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 786/1583] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.023247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.034743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 798/1584] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:44 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:45.035572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.035708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a22a6-dcf9-47d0-b2fc-599947bf170d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.056110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.066752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.101913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.102597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.102877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.103152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] PATH_INFO: `/v3/projects/caf4c294b65d453d8d0b9cd6fcd76976/users/914bea2efd334e6c8dc31698dfabbf8e/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.109713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a4d18-3c66-442e-87a3-b5bb4ab05638 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.115297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.116030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.116395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.116762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.117522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.118483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caf4c294b65d453d8d0b9cd6fcd76976, user_id=914bea2efd334e6c8dc31698dfabbf8e, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.119515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.121143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.131181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 799/1585] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.131983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.140586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 787/1586] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/caf4c294b65d453d8d0b9cd6fcd76976/users/914bea2efd334e6c8dc31698dfabbf8e/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.141388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.143574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046e6449-febd-4cb4-80ff-7f53ba4c7b73 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.203225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffaf139-c916-4948-bcc9-37a88a9a51a6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.204762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.205379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.205689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.205968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.206585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.207907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.208523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.208825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.209193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.209579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.210211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.212307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.234849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 788/1587] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:45.235800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.236922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 800/1588] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:45.237817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.249478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4ba04-bd24-4a5f-8d5e-1980f889884d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.300382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8861fc-6612-4d17-88a1-b9425033238b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.305020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.305729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.306004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.306331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.306971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.308560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.309863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.310471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.310747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.311166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.312309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.314389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.319016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 801/1589] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.319835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.329197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3f5454-db78-4618-bc3d-92d7b8928f14 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.334397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.334999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.335296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.335545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.336189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.337822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.345556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 789/1590] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/users => generated 285 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:45.345556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.349497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 802/1591] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.350387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.359472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3511dd80-e1d6-4ed7-97e7-1659e496be1c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.360921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161ce00-17e6-4311-9650-0f2ca87723f1 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.364163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.364851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.365166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.365464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.366169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.366289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4b5a-f6ca-4f1a-8cdc-69e85fdb581f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.366888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4b5a-f6ca-4f1a-8cdc-69e85fdb581f admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.367172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4b5a-f6ca-4f1a-8cdc-69e85fdb581f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.367446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4b5a-f6ca-4f1a-8cdc-69e85fdb581f admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.368083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4b5a-f6ca-4f1a-8cdc-69e85fdb581f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.368200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.369691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4b5a-f6ca-4f1a-8cdc-69e85fdb581f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.382456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 803/1592] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.384019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.384306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.384608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.392301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 790/1593] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:45.393131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.399428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.403238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341ec61f-bdc5-42ff-8f88-adae85213eda admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.406951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:45.408523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.409608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.410443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.410896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.411193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.411834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.413428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.416088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 804/1594] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:45.416842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.426952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42587c3-8d09-46ff-abf5-6a06e0d0153c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.427343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 791/1595] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.428136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.434115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.434796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.435093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.435359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.435987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.438531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b0063d-a70b-405b-b133-55c622967d12 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.438675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.443051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.443699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.443976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.444248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] PATH_INFO: `/v3/projects/d782d49821314cff8383836c3f0a43df/users/513103b63a344e63afadcdd452e21015/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.448684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.460234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d782d49821314cff8383836c3f0a43df, user_id=513103b63a344e63afadcdd452e21015, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.461114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 805/1596] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/users => generated 289 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:45.461973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.462939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.471528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda15e2-a996-439b-8b3f-4e2753a6a2ce admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.476008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 792/1597] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/d782d49821314cff8383836c3f0a43df/users/513103b63a344e63afadcdd452e21015/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.476810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.533210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.533851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.534131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.534397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.535038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.536629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.539528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d149bc44-154c-4304-a189-08817d7cc1f6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.544192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.544846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.545129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.545456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] PATH_INFO: `/v3/projects/3726a58288d9455b97e041a2e92f8850/users/3d8467cc62d948038690af1fdf5bb96a/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.549597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 806/1598] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.549597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.557540 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fbefaf-8a3c-468a-91d7-2b618c71d9a5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.562180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.562817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.563097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.563371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] PATH_INFO: `/v3/projects/47156a6d045b4147a837fbf132421a53/users/0b18352a23444324b27d17d2cee636fc/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.565124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3726a58288d9455b97e041a2e92f8850, user_id=3d8467cc62d948038690af1fdf5bb96a, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.567998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.578719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47156a6d045b4147a837fbf132421a53, user_id=0b18352a23444324b27d17d2cee636fc, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.581616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.581735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 793/1599] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/3726a58288d9455b97e041a2e92f8850/users/3d8467cc62d948038690af1fdf5bb96a/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.582473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.597159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 807/1600] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/47156a6d045b4147a837fbf132421a53/users/0b18352a23444324b27d17d2cee636fc/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.601842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.646631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47935409-9620-4b66-b744-c88e8ff7d119 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.657490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d1e51-a85d-4cf9-b3e6-115abe0b5b39 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.661501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.662148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.662455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.662726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.707275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.707886 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.708162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.708432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.709060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.710727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.712055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.728141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 794/1601] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.728739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.728935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.730212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.739779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147c066-c2bf-439a-87e2-79aa909c56a9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.740620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 808/1602] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:45.741395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.744872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d9ca19-cd28-4ae8-b7a4-b4f7248f31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.745504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9ca19-cd28-4ae8-b7a4-b4f7248f31cf admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.745824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9ca19-cd28-4ae8-b7a4-b4f7248f31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.746161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d9ca19-cd28-4ae8-b7a4-b4f7248f31cf admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.746744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9ca19-cd28-4ae8-b7a4-b4f7248f31cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.748352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d9ca19-cd28-4ae8-b7a4-b4f7248f31cf admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.752006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d052-540d-40aa-9d1b-6328620b06ef service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.756670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.757271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.757553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.757853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.758792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.762479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 795/1603] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.762642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.762751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.762751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.762751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.773362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 809/1604] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.774328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.781885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:45.785660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd432bfc-53c8-4be1-9bdd-df75e495ef65 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.792595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af7abce-5741-4bd3-bdfa-52f0bd43819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.793203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7abce-5741-4bd3-bdfa-52f0bd43819c admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.793484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7abce-5741-4bd3-bdfa-52f0bd43819c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.793796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af7abce-5741-4bd3-bdfa-52f0bd43819c admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.794432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7abce-5741-4bd3-bdfa-52f0bd43819c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.796676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af7abce-5741-4bd3-bdfa-52f0bd43819c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.800698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] MFA Rules not processed for user `513103b63a344e63afadcdd452e21015`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:45.802047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.808131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 810/1605] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.809407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dc6f7-9ebb-45eb-954a-1f38f39f384f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.809715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dc6f7-9ebb-45eb-954a-1f38f39f384f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.810003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dc6f7-9ebb-45eb-954a-1f38f39f384f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.829910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-0d9dc6f7-9ebb-45eb-954a-1f38f39f384f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:45.831252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0d9dc6f7-9ebb-45eb-954a-1f38f39f384f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.844016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 811/1606] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:45.845526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.845860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.846139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.867826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:45.869451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:45.879573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 812/1607] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:45.880917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.884614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 796/1608] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:45.888380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.896211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89b76c3-1f49-419f-8080-1dfe6d32a8e4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.900245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a74c3d-504f-465f-a7b4-ce6e1ab5f9e8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.901791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.902559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.902857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.903219 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.903964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.905505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.905887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.906647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.907002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.907383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] PATH_INFO: `/v3/projects/57dfc5e65dc1412ab199fff503acbfc3/users/c8b7cb87296a4ecea0b9ab44a0cbfd2c/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.916308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 813/1609] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:45.917109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.925215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57dfc5e65dc1412ab199fff503acbfc3, user_id=c8b7cb87296a4ecea0b9ab44a0cbfd2c, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.927423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de4d4a-734d-48e8-9a36-2df25dc1f201 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:45.928251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.932403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:45.933294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:45.933411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:45.933685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] PATH_INFO: `/v3/projects/a7a78e6a6010436b94e70eae6285bc24/users/334e4ae2fc654ee2a447be1f049c8746/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:45.950425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 797/1610] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/57dfc5e65dc1412ab199fff503acbfc3/users/c8b7cb87296a4ecea0b9ab44a0cbfd2c/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.950425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:45.954471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a78e6a6010436b94e70eae6285bc24, user_id=334e4ae2fc654ee2a447be1f049c8746, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:45.957373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:45.975161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 814/1611] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/a7a78e6a6010436b94e70eae6285bc24/users/334e4ae2fc654ee2a447be1f049c8746/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:45.975711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.026422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273b81b-640f-4dbf-8604-c55356254471 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.044168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795aa1f2-fa73-482b-94d5-0b3aa029760d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.050963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.053482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.053482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.053482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.092498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:46.101675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.102492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.102774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.103441 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] PATH_INFO: `/v3/projects/f4e9d8728a874a9aae02f347cde0467f/users/8c0f1c3464114a19963d956c66046018/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.124863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4e9d8728a874a9aae02f347cde0467f, user_id=8c0f1c3464114a19963d956c66046018, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.129175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.129715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.131156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.143488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 815/1612] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:46.144580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.157746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 798/1613] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:45 2022] PUT /identity/v3/projects/f4e9d8728a874a9aae02f347cde0467f/users/8c0f1c3464114a19963d956c66046018/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:46.157746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.158388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a952bf40-3269-4d88-828a-0d5395aa6805 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.159403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:46.226560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:46.234207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ffbe0-d48f-461e-b684-d25be0e72f36 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.238532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.239643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.240421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.240543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3caea6-7ca1-45b5-ab05-7fdc6f1e7e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.240888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.241110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3caea6-7ca1-45b5-ab05-7fdc6f1e7e41 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.241546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3caea6-7ca1-45b5-ab05-7fdc6f1e7e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.241854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3caea6-7ca1-45b5-ab05-7fdc6f1e7e41 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.242690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3caea6-7ca1-45b5-ab05-7fdc6f1e7e41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.245028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3caea6-7ca1-45b5-ab05-7fdc6f1e7e41 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.253773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.255637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.275054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 799/1614] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:46.275889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.286661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c0b69-b6ce-4715-8560-3e321e820e6f service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.292305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.293157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.293342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.293704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] PATH_INFO: `/v3/projects/e35cf25979d449219f522afe907f1fb8/users/d89b82ec277f4139a80724a975d74f22/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.304741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 816/1615] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:46 2022] POST /identity/v3/projects => generated 373 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:46.306659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317bfa-0814-415e-8cbb-b2ccd7aaa951 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.307471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317bfa-0814-415e-8cbb-b2ccd7aaa951 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.307988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52317bfa-0814-415e-8cbb-b2ccd7aaa951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.313919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35cf25979d449219f522afe907f1fb8, user_id=d89b82ec277f4139a80724a975d74f22, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.317082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.336912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-52317bfa-0814-415e-8cbb-b2ccd7aaa951 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:46.339445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-52317bfa-0814-415e-8cbb-b2ccd7aaa951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:46.352945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 800/1616] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:46 2022] PUT /identity/v3/projects/e35cf25979d449219f522afe907f1fb8/users/d89b82ec277f4139a80724a975d74f22/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:46.353676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.440601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 817/1617] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:46.442331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd9126-d726-4d51-916c-fbe365d13146 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.442621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd9126-d726-4d51-916c-fbe365d13146 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.442944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bd9126-d726-4d51-916c-fbe365d13146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.485028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f6bd9126-d726-4d51-916c-fbe365d13146 None None] MFA Rules not processed for user `8c0f1c3464114a19963d956c66046018`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:46.486270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f6bd9126-d726-4d51-916c-fbe365d13146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:46.502472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434ce46b-c8d6-488b-b016-51e3e9ccab3a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.508466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.509235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.509700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.510156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.511065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.513037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.525317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 801/1618] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:46.526331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.542056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0198e71-2a13-457e-b65f-a5d5cdcf97d9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.548362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.549484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.550157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.550403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.551473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.555276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.576152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 818/1619] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:46.578404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.578695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.578972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.597807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 802/1620] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:46 2022] POST /identity/v3/users => generated 293 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:46.598953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.687025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] MFA Rules not processed for user `d89b82ec277f4139a80724a975d74f22`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:46.695841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:46.740200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:46.776575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254dab6c-e3b3-47fa-909d-9679cbc8a16a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.787187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.787784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.788061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.788330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.816741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.818599 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.846058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 803/1621] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:46.846706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:46.847221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.856465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 819/1622] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:46.862024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.875535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddbdcf4-cd4d-4104-a784-6b30fb8333b9 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.882881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7013f5b-0604-4e7b-979b-947cbb0d7c19 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:46.887344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.888027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.888523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.888647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] PATH_INFO: `/v3/projects/a7a78e6a6010436b94e70eae6285bc24/users/334e4ae2fc654ee2a447be1f049c8746/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.893129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7a78e6a6010436b94e70eae6285bc24, user_id=334e4ae2fc654ee2a447be1f049c8746, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.898138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae3ed5-1543-4d6a-8c6f-ac924e7c934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:46.898867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae3ed5-1543-4d6a-8c6f-ac924e7c934f admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.899099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae3ed5-1543-4d6a-8c6f-ac924e7c934f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.899359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae3ed5-1543-4d6a-8c6f-ac924e7c934f admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.900068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae3ed5-1543-4d6a-8c6f-ac924e7c934f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:46.903501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.905980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae3ed5-1543-4d6a-8c6f-ac924e7c934f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:46.926301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 820/1623] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:46 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:46.927771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b0671-a1f6-472c-a23f-5c6d882f5eed None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:46.928054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b0671-a1f6-472c-a23f-5c6d882f5eed None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:46.928327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b0671-a1f6-472c-a23f-5c6d882f5eed None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:46.932457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 804/1624] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:46 2022] PUT /identity/v3/projects/a7a78e6a6010436b94e70eae6285bc24/users/334e4ae2fc654ee2a447be1f049c8746/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:46.933770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:46.940204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 821/1625] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 05:45:46 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:45:46.941150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b0671-a1f6-472c-a23f-5c6d882f5eed None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.064693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79601a7-d8fc-4eaa-bb44-38c00cbe0571 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.071849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7371fb0-5b0f-4d6c-b262-55ab4e455c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.072518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7371fb0-5b0f-4d6c-b262-55ab4e455c6f admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.072809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7371fb0-5b0f-4d6c-b262-55ab4e455c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.073270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7371fb0-5b0f-4d6c-b262-55ab4e455c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.074321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7371fb0-5b0f-4d6c-b262-55ab4e455c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.076389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7371fb0-5b0f-4d6c-b262-55ab4e455c6f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.112224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b0671-a1f6-472c-a23f-5c6d882f5eed None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.118267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00e632-7c73-4355-ba1a-a2df6cff731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.118700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 805/1626] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:46 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:47.119107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e632-7c73-4355-ba1a-a2df6cff731b admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.119399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e632-7c73-4355-ba1a-a2df6cff731b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.119678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e632-7c73-4355-ba1a-a2df6cff731b admin admin] PATH_INFO: `/v3/projects/0560ad744a4a487d89517896293deaf7/users/45aa13b000444b299e5651dc7a32b811/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.120113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.120792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.121280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.156192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e632-7c73-4355-ba1a-a2df6cff731b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560ad744a4a487d89517896293deaf7, user_id=45aa13b000444b299e5651dc7a32b811, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.159945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e632-7c73-4355-ba1a-a2df6cff731b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.183413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:47.184722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:47.197319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 822/1627] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:46 2022] PUT /identity/v3/projects/0560ad744a4a487d89517896293deaf7/users/45aa13b000444b299e5651dc7a32b811/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:47.200665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.201154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.202710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.287292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] MFA Rules not processed for user `334e4ae2fc654ee2a447be1f049c8746`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:47.289256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:47.308287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 806/1628] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 05:45:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:47.309322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.361511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:47.410291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda5fdd-9134-4318-8e82-6e6eafef60c4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.412294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 823/1629] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:47 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:47.413381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.428105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a60a4c-f5cf-4409-9f9b-0b5fa8426850 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.429774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.430993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.431597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.432379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.433631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.433834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.434231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.434524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.434840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.435509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.437919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.438115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.454937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 824/1630] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:47.456416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.488861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39707b8-4809-4fca-a184-c5ce955c99f3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.490869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 807/1631] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:47 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:47.491844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.505603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.506800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.507429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.508041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.612374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:47.634998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.638105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.656056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7e8805-c00d-4383-9f3f-4e9944a4ea54 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.657086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 825/1632] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:47.659325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.671510 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220014a9-a841-45ff-a76e-057aeb810cf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.672250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220014a9-a841-45ff-a76e-057aeb810cf1 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.672600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220014a9-a841-45ff-a76e-057aeb810cf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.673052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220014a9-a841-45ff-a76e-057aeb810cf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.677636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1891f0-01ea-4b2e-a428-deb774b50356 service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.685588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.687473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.688234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.688577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] PATH_INFO: `/v3/projects/0560ad744a4a487d89517896293deaf7/users/45aa13b000444b299e5651dc7a32b811/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.691844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220014a9-a841-45ff-a76e-057aeb810cf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.692269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0560ad744a4a487d89517896293deaf7, user_id=45aa13b000444b299e5651dc7a32b811, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.693891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220014a9-a841-45ff-a76e-057aeb810cf1 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.697753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.728508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 826/1633] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:47 2022] PUT /identity/v3/projects/0560ad744a4a487d89517896293deaf7/users/45aa13b000444b299e5651dc7a32b811/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:47.728508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.728857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 808/1634] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:47.731176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.731568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.732071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.794918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] MFA Rules not processed for user `45aa13b000444b299e5651dc7a32b811`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:47.796989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:47.884454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957cdaf3-b23c-42ed-89ef-3bf04a073be4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.898021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.898658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.898941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.899295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.902835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:47.906005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:47.943273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 827/1635] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:47.946155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:47.964836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36705eba-1129-40e4-a10b-00ce94ae85a7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:47.975039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:47.976940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:47.977213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:47.977490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] PATH_INFO: `/v3/projects/51e0ae07d5c84e29b85bbb01f0c7c030/users/c9310661b957439fad664e1b5a0da8ac/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:47.981650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 809/1636] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:47.991957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.019557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e0ae07d5c84e29b85bbb01f0c7c030, user_id=c9310661b957439fad664e1b5a0da8ac, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.028016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.063493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 828/1637] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:47 2022] PUT /identity/v3/projects/51e0ae07d5c84e29b85bbb01f0c7c030/users/c9310661b957439fad664e1b5a0da8ac/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:48.064445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.148791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:48.156216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2fac2d-f94b-46c9-945a-a4576d52af20 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.168236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e5c503-42b3-4a01-8a82-3c85132cc76e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.181609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.181609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.181609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.181609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.182248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.182248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.194719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 829/1638] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:48.204053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.224880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88566d-d250-46ef-ab5d-4bb7098ccfa0 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.231759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f4583-6a00-429a-a77e-3f80db13f94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.233468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4583-6a00-429a-a77e-3f80db13f94b admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.233824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4583-6a00-429a-a77e-3f80db13f94b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.234218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f4583-6a00-429a-a77e-3f80db13f94b admin admin] PATH_INFO: `/v3/projects/51e0ae07d5c84e29b85bbb01f0c7c030/users/c9310661b957439fad664e1b5a0da8ac/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.241943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f4583-6a00-429a-a77e-3f80db13f94b admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e0ae07d5c84e29b85bbb01f0c7c030, user_id=c9310661b957439fad664e1b5a0da8ac, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.250063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f4583-6a00-429a-a77e-3f80db13f94b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.275612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 830/1639] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:48 2022] PUT /identity/v3/projects/51e0ae07d5c84e29b85bbb01f0c7c030/users/c9310661b957439fad664e1b5a0da8ac/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:48.277280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c962d-21ac-4964-b529-3dc7b806fe93 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.277599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c962d-21ac-4964-b529-3dc7b806fe93 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.277861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559c962d-21ac-4964-b529-3dc7b806fe93 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.279514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504d263-63a8-4a73-8142-ce2aadb6526e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.279748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 831/1640] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 05:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:45:48.280369 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504d263-63a8-4a73-8142-ce2aadb6526e service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.280723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.280850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504d263-63a8-4a73-8142-ce2aadb6526e service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.281010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.281299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.281417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504d263-63a8-4a73-8142-ce2aadb6526e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.327992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] MFA Rules not processed for user `c9310661b957439fad664e1b5a0da8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:48.328977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:48.345066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:48.430071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504d263-63a8-4a73-8142-ce2aadb6526e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.432424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504d263-63a8-4a73-8142-ce2aadb6526e service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.465049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 810/1641] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:48.466551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.471420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.472408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 832/1642] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:48.472925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.510410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.542184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:48.545119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:48.650328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95198d6-e043-4344-9ac3-f3a1a60c8670 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.654306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.655124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.655252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.655505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.676088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 811/1643] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:45:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:48.763153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.775792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.781913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.814703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 833/1644] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:48.832906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:48.833100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:48.857769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac16cec-49ac-409e-b496-63722b2f49bf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.861902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db53c8-cc73-4591-8c39-8dac0ff45271 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:48.866208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.868419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.869223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.870196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] PATH_INFO: `/v3/users/fd2ee5dfc7a54e39b940f22da1ae9d3d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.874466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866716a1-fbf6-49bd-9d10-ae1151ca913d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:48.874638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2ee5dfc7a54e39b940f22da1ae9d3d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.876259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866716a1-fbf6-49bd-9d10-ae1151ca913d service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:48.878049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866716a1-fbf6-49bd-9d10-ae1151ca913d service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:48.881497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.881633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866716a1-fbf6-49bd-9d10-ae1151ca913d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:48.908984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866716a1-fbf6-49bd-9d10-ae1151ca913d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:48.911533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866716a1-fbf6-49bd-9d10-ae1151ca913d service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:48.946695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 834/1645] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:48.985167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2ee5dfc7a54e39b940f22da1ae9d3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:48.986178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2ee5dfc7a54e39b940f22da1ae9d3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.005691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2ee5dfc7a54e39b940f22da1ae9d3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.007248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2ee5dfc7a54e39b940f22da1ae9d3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.031137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2ee5dfc7a54e39b940f22da1ae9d3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:49.068988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 812/1646] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:48 2022] DELETE /identity/v3/users/fd2ee5dfc7a54e39b940f22da1ae9d3d => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:49.071018 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.071990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.072510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.115342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:49.118591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.262900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:49.296627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 835/1647] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:49.314010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:49.315568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:49.474788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bea00f-982e-4c11-b9f0-39e06835f6bc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:49.481754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df09623-868d-4492-8423-ff1be2101c21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:49.483624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df09623-868d-4492-8423-ff1be2101c21 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:49.484730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df09623-868d-4492-8423-ff1be2101c21 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:49.485728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df09623-868d-4492-8423-ff1be2101c21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:49.499132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df09623-868d-4492-8423-ff1be2101c21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:49.500853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df09623-868d-4492-8423-ff1be2101c21 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:49.535973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 813/1648] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:50.492531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.494363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.516864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4af12f-0801-496c-92d0-adb86c6109dc None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:50.530206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:50.531075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.531482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.532118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] PATH_INFO: `/v3/users/513103b63a344e63afadcdd452e21015` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.533430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] RBAC: Authorizing `identity:delete_user(user_id=513103b63a344e63afadcdd452e21015)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:50.537201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: WARNING py.warnings [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:45:50.537201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: warnings.warn(msg) Aug 24 05:45:50.537201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: Aug 24 05:45:50.539140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:50.631558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ce944-b811-44ac-8379-ba2238059aba None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.632140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ce944-b811-44ac-8379-ba2238059aba None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.632140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ce944-b811-44ac-8379-ba2238059aba None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.633446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 814/1649] 10.222.0.73 () {54 vars in 962 bytes} [Wed Aug 24 05:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:45:50.647794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61b14-c086-4020-813e-2dfd4f5bdaab None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.648384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61b14-c086-4020-813e-2dfd4f5bdaab None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.648666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed61b14-c086-4020-813e-2dfd4f5bdaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.688459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '513103b63a344e63afadcdd452e21015'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.689067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '513103b63a344e63afadcdd452e21015'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.698873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '513103b63a344e63afadcdd452e21015'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.699339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0ed61b14-c086-4020-813e-2dfd4f5bdaab None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:50.699655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '513103b63a344e63afadcdd452e21015'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.703947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0ed61b14-c086-4020-813e-2dfd4f5bdaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.726762 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cde0fbac-fcb2-409b-8830-83d3ce1c643b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '513103b63a344e63afadcdd452e21015'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:50.749019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 836/1650] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:50 2022] DELETE /identity/v3/users/513103b63a344e63afadcdd452e21015 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:50.751400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.751754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.752040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.797600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:50.798591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.829158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:50.858414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 815/1651] 10.222.0.73 () {58 vars in 1059 bytes} [Wed Aug 24 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:50.860057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5a2a6-f58f-42e8-8207-c7c05a0b98a5 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.860540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5a2a6-f58f-42e8-8207-c7c05a0b98a5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.861275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5a2a6-f58f-42e8-8207-c7c05a0b98a5 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.862654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 816/1652] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 05:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:45:50.868301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:50.868584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:50.868867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:50.911344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 837/1653] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:50.911537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:50.911833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:50.934564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:50.937019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:51.058547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 817/1654] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:45:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:51.069027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80889c-9a4b-4e3a-b1a9-2840530245b6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:51.072850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:51.074494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:51.081509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:51.082186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:51.082474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:51.082758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:51.106140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:51.107602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:51.149278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 838/1655] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:51.216997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc9417-cad4-4c7f-811d-3c34811067e5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:51.230940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:51.233301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:51.235046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:51.236911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:51.290209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:51.290209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:51.322471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 818/1656] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:52.511022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:52.513623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:52.549640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8fac7b-c902-4c68-9ac9-187c3dcd1ea2 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:52.561993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:52.562662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:52.562941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:52.563209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] PATH_INFO: `/v3/projects/d12cf60eb26941e6a9b1dcfe56b3fa9e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:52.570567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12cf60eb26941e6a9b1dcfe56b3fa9e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:52.572538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:52.575466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:52.719157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12cf60eb26941e6a9b1dcfe56b3fa9e', 'request_id': 'req-77958c50-d5fc-48e0-860e-1d02bc8899a7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.758794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12cf60eb26941e6a9b1dcfe56b3fa9e', 'request_id': 'req-77958c50-d5fc-48e0-860e-1d02bc8899a7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.796738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12cf60eb26941e6a9b1dcfe56b3fa9e', 'request_id': 'req-77958c50-d5fc-48e0-860e-1d02bc8899a7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.828701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] The token cache is being invalidate because project d12cf60eb26941e6a9b1dcfe56b3fa9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:52.828996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.837838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:52.839563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 839/1657] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:52 2022] DELETE /identity/v3/projects/d12cf60eb26941e6a9b1dcfe56b3fa9e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:52.845304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:52.857692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fa8adc-b18b-4a94-b3aa-706bfaf8b680 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:52.950145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:52.950809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:52.951097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:52.951513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] PATH_INFO: `/v3/users/334e4ae2fc654ee2a447be1f049c8746` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:52.956990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334e4ae2fc654ee2a447be1f049c8746)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:52.957353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:53.015228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:53.031189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334e4ae2fc654ee2a447be1f049c8746'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.032729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334e4ae2fc654ee2a447be1f049c8746'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.077708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334e4ae2fc654ee2a447be1f049c8746'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.079801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334e4ae2fc654ee2a447be1f049c8746'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.102389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334e4ae2fc654ee2a447be1f049c8746'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.156329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:53.158004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:53.169457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 819/1658] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:52 2022] DELETE /identity/v3/users/334e4ae2fc654ee2a447be1f049c8746 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:53.199629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77958c50-d5fc-48e0-860e-1d02bc8899a7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:53.336133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:53.340750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.341088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.341371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] PATH_INFO: `/v3/projects/d782d49821314cff8383836c3f0a43df` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.343680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d782d49821314cff8383836c3f0a43df)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:53.346307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:53.409749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd782d49821314cff8383836c3f0a43df', 'request_id': 'req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.420185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd782d49821314cff8383836c3f0a43df', 'request_id': 'req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.441079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd782d49821314cff8383836c3f0a43df', 'request_id': 'req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.461414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] The token cache is being invalidate because project d782d49821314cff8383836c3f0a43df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:53.461906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.462727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.471073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:53.472949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:53.474697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 840/1659] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:53 2022] DELETE /identity/v3/projects/d782d49821314cff8383836c3f0a43df => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:45:53.494609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56427745-95dd-4dcc-bd81-fd0a4d57f0f9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:53.594980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:53.595659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:53.595962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:53.596461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] PATH_INFO: `/v3/users/45aa13b000444b299e5651dc7a32b811` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:53.597796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] RBAC: Authorizing `identity:delete_user(user_id=45aa13b000444b299e5651dc7a32b811)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:53.601884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:53.702089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45aa13b000444b299e5651dc7a32b811'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.705326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45aa13b000444b299e5651dc7a32b811'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.729391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45aa13b000444b299e5651dc7a32b811'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.732004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45aa13b000444b299e5651dc7a32b811'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.749734 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2ff29466-f789-4719-a03e-820430ebc13a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45aa13b000444b299e5651dc7a32b811'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:53.787699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 820/1660] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:53 2022] DELETE /identity/v3/users/45aa13b000444b299e5651dc7a32b811 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:54.742703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:54.743789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:54.769052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f4aa53-0b45-46d6-9422-e7bcb3c2bdc0 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:54.845323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:54.980970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:54.984562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:54.984868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:54.992442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] PATH_INFO: `/v3/projects/a7a78e6a6010436b94e70eae6285bc24` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:54.994811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7a78e6a6010436b94e70eae6285bc24)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:54.996751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:55.169011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7a78e6a6010436b94e70eae6285bc24', 'request_id': 'req-650dee8b-8b34-4611-af5b-80e8d78b4e00'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:55.198366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7a78e6a6010436b94e70eae6285bc24', 'request_id': 'req-650dee8b-8b34-4611-af5b-80e8d78b4e00'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:55.211224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7a78e6a6010436b94e70eae6285bc24', 'request_id': 'req-650dee8b-8b34-4611-af5b-80e8d78b4e00'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:55.252490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] The token cache is being invalidate because project a7a78e6a6010436b94e70eae6285bc24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:55.252995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:55.258546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-650dee8b-8b34-4611-af5b-80e8d78b4e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:55.261965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 841/1661] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:54 2022] DELETE /identity/v3/projects/a7a78e6a6010436b94e70eae6285bc24 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:55.267765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:55.268301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:55.268301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:55.314800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:55.316078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:55.408345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:55.485649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 821/1662] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:55.488043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256c8d4-515f-4549-8906-481066409f60 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:55.489431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256c8d4-515f-4549-8906-481066409f60 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:55.489777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b256c8d4-515f-4549-8906-481066409f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:55.558294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b256c8d4-515f-4549-8906-481066409f60 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:55.560709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b256c8d4-515f-4549-8906-481066409f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:55.599506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 842/1663] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:55.603907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:55.635031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fe5889-ec46-4bb5-a4d7-9e8e9f7fca5a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:55.650370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:55.651233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:55.651434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:55.651687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:55.662083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:55.662422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:55.664124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:55.699151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 822/1664] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:55.705814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256c8d4-515f-4549-8906-481066409f60 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:55.721165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b256c8d4-515f-4549-8906-481066409f60 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:55.728148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:55.728885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:55.729172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:55.729448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:55.731244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:55.733826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:55.747526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:55.750132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:55.772485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fdf1d7-e404-4312-a540-8a6b56c5d48b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:55.782275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:55.782933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:55.783220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:55.783500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] PATH_INFO: `/v3/projects/0560ad744a4a487d89517896293deaf7` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:55.794507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] RBAC: Authorizing `identity:delete_project(project_id=0560ad744a4a487d89517896293deaf7)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:55.796200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: WARNING py.warnings [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 05:45:55.796200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: warnings.warn(msg) Aug 24 05:45:55.796200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: Aug 24 05:45:55.797603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:55.818509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 843/1665] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:55 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:55.822747 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:55.947960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0560ad744a4a487d89517896293deaf7', 'request_id': 'req-adeebea4-408c-47cd-9293-d56be76624be'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:55.968963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0560ad744a4a487d89517896293deaf7', 'request_id': 'req-adeebea4-408c-47cd-9293-d56be76624be'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:56.001025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b9e43-d4f8-4819-9124-c5ab35951bc0 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.011643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0560ad744a4a487d89517896293deaf7', 'request_id': 'req-adeebea4-408c-47cd-9293-d56be76624be'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:56.033784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] The token cache is being invalidate because project 0560ad744a4a487d89517896293deaf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:56.034099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:56.038001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-adeebea4-408c-47cd-9293-d56be76624be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:56.044760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 823/1666] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:55 2022] DELETE /identity/v3/projects/0560ad744a4a487d89517896293deaf7 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:56.050166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684cec8-635e-41dd-8404-511f135dd3ef None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.050455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684cec8-635e-41dd-8404-511f135dd3ef None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.050738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f684cec8-635e-41dd-8404-511f135dd3ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.077192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f684cec8-635e-41dd-8404-511f135dd3ef None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:56.078860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f684cec8-635e-41dd-8404-511f135dd3ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.083987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:56.160699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.163308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.164124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.164430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.165239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.169085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.188374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 824/1667] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:45:56.203240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.203639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.203932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.241635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 844/1668] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:55 2022] POST /identity/v3/users => generated 283 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:56.243380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.250292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.266219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:56.267733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.303474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 825/1669] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:56.304392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.311875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.347491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac7296-4694-4177-8ef4-731e447aa3e2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.353434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.354237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.354526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.354804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.355609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.357267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.389824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 826/1670] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:56.392882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.412779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f92de55-e0af-4d01-aa56-12553cc1e9c0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.421016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.422105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.423192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.423669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.437348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:45:56.437838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.440750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.469030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 827/1671] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:45:56.470696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.472781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620b28e-4d43-450d-9ae1-add36cbdf65e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.489481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96dbc01-6b37-4fc2-b7b6-c2ed6aaaac4d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.495849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.496277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.496522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.496956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.497254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.497549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.498339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.498924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.499271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.501398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.502557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.519018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 828/1672] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:56.519018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.522151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:56.528474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2376ff16-9785-4d21-a76e-56748064b82e admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.544688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.544688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.544949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.548088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.548818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.558109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.622126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 829/1673] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:56 2022] POST /identity/v3/projects => generated 399 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:56.627396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.743493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.746257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.776348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 845/1674] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:56.790775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:56.795621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:56.819807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36057eb4-afde-4cc0-9bcc-1b53c450a030 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.826042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.827895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.828802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.829589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] PATH_INFO: `/v3/projects/4c9d6530be4346fe9bde87d62026017d/users/bc28b3b2a48e416ebbaa0a814baa2626/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.846302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c118481-5eeb-4a8a-adbe-6c843ced9be0 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:56.862637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9d6530be4346fe9bde87d62026017d, user_id=bc28b3b2a48e416ebbaa0a814baa2626, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.863820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:56.864482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:56.864763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:56.865037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] PATH_INFO: `/v3/users/8c0f1c3464114a19963d956c66046018` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:56.868511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.869706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0f1c3464114a19963d956c66046018)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:56.878430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:56.905462 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 830/1675] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:56 2022] PUT /identity/v3/projects/4c9d6530be4346fe9bde87d62026017d/users/bc28b3b2a48e416ebbaa0a814baa2626/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:56.907299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.036024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0f1c3464114a19963d956c66046018'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:57.036411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0f1c3464114a19963d956c66046018'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:57.066102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0f1c3464114a19963d956c66046018'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:57.068061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0f1c3464114a19963d956c66046018'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:57.094575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:57.098222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407c160-038a-4a4f-a65d-8c7d1866da10 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.105542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-0c003d41-8ca9-4457-9a55-79e2c17dca1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0f1c3464114a19963d956c66046018'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:57.113338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837a54f-2cb5-4984-a9f2-5b7b703ab9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.113962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837a54f-2cb5-4984-a9f2-5b7b703ab9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.114928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837a54f-2cb5-4984-a9f2-5b7b703ab9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.115534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837a54f-2cb5-4984-a9f2-5b7b703ab9e8 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.116698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837a54f-2cb5-4984-a9f2-5b7b703ab9e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.120214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837a54f-2cb5-4984-a9f2-5b7b703ab9e8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.146883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 846/1676] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:45:56 2022] DELETE /identity/v3/users/8c0f1c3464114a19963d956c66046018 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:57.150742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.152917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.153215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.180022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 831/1677] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:56 2022] POST /identity/v3/users => generated 300 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:57.182915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.183071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.183310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.205207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] MFA Rules not processed for user `bc28b3b2a48e416ebbaa0a814baa2626`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:57.206539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:57.224466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:57.230892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:57.365362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:57.395537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 847/1678] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:57.396464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.399941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:57.424738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 832/1679] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:57.426505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae22cb-49c2-4f2d-ad24-256ff193954b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.436458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.441307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:57.451021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.452071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.452432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.452829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.453827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.456411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.492572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a290a7-959d-4cc1-a471-268217d4271c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.494228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 848/1680] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:57.495878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.500860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:57.524729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.527059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.527589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.530566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.532416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.550882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.653184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 833/1681] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:57 2022] POST /identity/v3/projects => generated 363 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:57.654142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.666722 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c482a3-cb5e-43c1-905a-439d22f4a410 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.792694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.793453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.794993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.795244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.799216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776992a-0993-4d43-b640-043dd272cb92 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.812971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.813655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.813924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.814162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.815431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.816972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.818155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.819861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.844620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 849/1682] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:57.845547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.845913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 834/1683] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:57.848307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.871560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d0430-7205-4b63-975c-20d844b2c897 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.872425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18613d-b4ae-45a3-850e-794fc42c12d9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:57.879026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bc7cb-e94d-401b-b2f2-cbdfea10c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.880447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc7cb-e94d-401b-b2f2-cbdfea10c722 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.880447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc7cb-e94d-401b-b2f2-cbdfea10c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.880447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bc7cb-e94d-401b-b2f2-cbdfea10c722 admin admin] PATH_INFO: `/v3/projects/003b4ddd274d42d49fc1e5d70a772505/users/d0842c2ec2f94d6680ccacc39840dda1/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.885923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:57.885923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.885923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.885923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.886378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.892861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.909735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc7cb-e94d-401b-b2f2-cbdfea10c722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=003b4ddd274d42d49fc1e5d70a772505, user_id=d0842c2ec2f94d6680ccacc39840dda1, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:57.915744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bc7cb-e94d-401b-b2f2-cbdfea10c722 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:57.936024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 850/1684] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:57 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:57.936999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:57.942923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 835/1685] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:57 2022] PUT /identity/v3/projects/003b4ddd274d42d49fc1e5d70a772505/users/d0842c2ec2f94d6680ccacc39840dda1/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:57.948709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:57.957313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:57.958208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:57.958920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:57.959277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd999-f2eb-4787-af27-f604931dbc2a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.047226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] MFA Rules not processed for user `d0842c2ec2f94d6680ccacc39840dda1`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:58.047415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:58.096963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.098184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.098678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.099169 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.100883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.103757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.118649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 851/1686] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:57 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:45:58.125104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.152109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f4690-cd06-41f3-883e-ba9256773087 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.159809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.160831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.161334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.161970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] PATH_INFO: `/v3/projects/99edf9fe9af64acbb4c6d1ed64797e50/users/ff5bbd6c562b44a9b25f32c9297b0e8d/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.182606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=99edf9fe9af64acbb4c6d1ed64797e50, user_id=ff5bbd6c562b44a9b25f32c9297b0e8d, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.189956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.204839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 836/1687] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:57 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:58.209601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.219100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 852/1688] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:58 2022] PUT /identity/v3/projects/99edf9fe9af64acbb4c6d1ed64797e50/users/ff5bbd6c562b44a9b25f32c9297b0e8d/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:58.224824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.237107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2142ff-b96b-4272-81b8-83be9f774f8d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.309668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:58.331513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fbcd7-d808-4741-9c8d-2e0ebdaf8e3d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.343633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.344578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.345045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.345467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.346346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.349155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.370974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 853/1689] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:58.376708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.391875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e936e-90c0-45e8-b3c4-972e22f6bc80 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.404898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e392e9e-d97e-4af6-89d9-98b5dab75cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.405967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e392e9e-d97e-4af6-89d9-98b5dab75cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.406323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e392e9e-d97e-4af6-89d9-98b5dab75cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.406911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e392e9e-d97e-4af6-89d9-98b5dab75cd8 admin admin] PATH_INFO: `/v3/projects/99edf9fe9af64acbb4c6d1ed64797e50/users/ff5bbd6c562b44a9b25f32c9297b0e8d/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.407208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.407842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.408125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.408399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.409095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.410078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e392e9e-d97e-4af6-89d9-98b5dab75cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99edf9fe9af64acbb4c6d1ed64797e50, user_id=ff5bbd6c562b44a9b25f32c9297b0e8d, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.410976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.413708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e392e9e-d97e-4af6-89d9-98b5dab75cd8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.447319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 854/1690] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:58 2022] PUT /identity/v3/projects/99edf9fe9af64acbb4c6d1ed64797e50/users/ff5bbd6c562b44a9b25f32c9297b0e8d/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:58.458272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.458635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.458823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.484439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 837/1691] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:45:58 2022] POST /identity/v3/projects => generated 397 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:58.490231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.549921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] MFA Rules not processed for user `ff5bbd6c562b44a9b25f32c9297b0e8d`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:58.556529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:58.651226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:58.697887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7767a803-4bb6-4520-886d-5153a67db039 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.714392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.715037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.715331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.715583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.716210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.718849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.733298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 855/1692] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:58.744800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.746423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:58.772801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 838/1693] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:45:58 2022] POST /identity/v3/users => generated 305 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:45:58.776485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.799530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79404ed0-a3c9-4d59-b053-fa371c6bd838 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.818391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.819196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.819578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.819952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.820804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.822582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.841298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 839/1694] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:58.845366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.863980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5758ab-c11b-4817-9f97-526393761302 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.877410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.878128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.878418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.878712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] PATH_INFO: `/v3/projects/704b86c87a9e4642b736c4f99c33b18e/users/b2590e437303451aa93e7ab5d5336026/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.893706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c0f919-a9f1-4208-be75-00f6824df170 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:58.905556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:58.906217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:58.906501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:58.906774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:58.918795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] RBAC: Authorizing `identity:create_grant(project_id=704b86c87a9e4642b736c4f99c33b18e, user_id=b2590e437303451aa93e7ab5d5336026, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.922069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.931665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:58.933651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:58.953922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 840/1695] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:58 2022] PUT /identity/v3/projects/704b86c87a9e4642b736c4f99c33b18e/users/b2590e437303451aa93e7ab5d5336026/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:58.957520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:58.977778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 856/1696] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:58 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:45:59.095802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:59.117378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630822e5-90f9-4411-84bb-c1d8330a1e5b service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:59.123442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e709b4d-f9a5-4bc0-b7d6-02f42da5860e admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:59.128885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:59.129905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.130427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.131037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.132017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:59.135598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:59.149155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 841/1697] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:45:58 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:45:59.153319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:59.172697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dfcded-62f1-4557-88e3-7d8774a2607d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:59.188791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c101f73-14fb-48de-b3f2-1427c15d8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:59.189632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c101f73-14fb-48de-b3f2-1427c15d8f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.190166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c101f73-14fb-48de-b3f2-1427c15d8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.190568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c101f73-14fb-48de-b3f2-1427c15d8f25 admin admin] PATH_INFO: `/v3/projects/704b86c87a9e4642b736c4f99c33b18e/users/b2590e437303451aa93e7ab5d5336026/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.199214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c101f73-14fb-48de-b3f2-1427c15d8f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704b86c87a9e4642b736c4f99c33b18e, user_id=b2590e437303451aa93e7ab5d5336026, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:59.202451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c101f73-14fb-48de-b3f2-1427c15d8f25 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:59.206669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:59.239380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 842/1698] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:45:59 2022] PUT /identity/v3/projects/704b86c87a9e4642b736c4f99c33b18e/users/b2590e437303451aa93e7ab5d5336026/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:59.249840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.250125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.250396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.306718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:59.308144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.308603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.308875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.322322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] MFA Rules not processed for user `b2590e437303451aa93e7ab5d5336026`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:59.323730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:59.462125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:59.463781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:59.486876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 843/1700] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:59.487783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:59.489887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 857/1700] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:59.498359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:59.499300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:59.500780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:59.531436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea67249-e497-4218-b061-d2c4e0c88fec service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:59.627855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb58cf2-8be5-427a-ab50-e243b393b096 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:45:59.640373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:45:59.646214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:59.646986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.647152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.647412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] PATH_INFO: `/v3/projects/f4e9d8728a874a9aae02f347cde0467f` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.648608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4e9d8728a874a9aae02f347cde0467f)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:59.650661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:59.663485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d550d8a-d5b2-42f7-bb52-97ad7a71495c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:45:59.665457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d550d8a-d5b2-42f7-bb52-97ad7a71495c service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.666717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d550d8a-d5b2-42f7-bb52-97ad7a71495c service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.667394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d550d8a-d5b2-42f7-bb52-97ad7a71495c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.684693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d550d8a-d5b2-42f7-bb52-97ad7a71495c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:45:59.688013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d550d8a-d5b2-42f7-bb52-97ad7a71495c service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:45:59.701700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 858/1701] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:45:59.703668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:45:59.704272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:45:59.704683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:45:59.776899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] MFA Rules not processed for user `0b18352a23444324b27d17d2cee636fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:45:59.781617 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:59.882235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4e9d8728a874a9aae02f347cde0467f', 'request_id': 'req-f1362558-2317-47b5-ae1a-e75bd77b0c8b'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:59.894199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4e9d8728a874a9aae02f347cde0467f', 'request_id': 'req-f1362558-2317-47b5-ae1a-e75bd77b0c8b'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:59.905660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 859/1702] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:45:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:45:59.906898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:59.931105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4e9d8728a874a9aae02f347cde0467f', 'request_id': 'req-f1362558-2317-47b5-ae1a-e75bd77b0c8b'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:59.958823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] The token cache is being invalidate because project f4e9d8728a874a9aae02f347cde0467f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:45:59.959280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:59.960593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:45:59.965656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 844/1703] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:45:59 2022] DELETE /identity/v3/projects/f4e9d8728a874a9aae02f347cde0467f => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:45:59.965656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:45:59.973080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:45:59.989893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:00.012528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588e09d-f4d0-4174-a0c5-a536de19ecf9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:00.025547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:00.026291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.026645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.026987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.030605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:00.113476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1362558-2317-47b5-ae1a-e75bd77b0c8b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:00.115038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:00.116745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624eb3bd-8dbf-4ed4-a303-931def6cefd2 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:00.129539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:00.130858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.131259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.131629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.137030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 860/1704] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:59 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:00.137342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:00.140076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.140546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.140983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.155241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:00.157290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28a33b-79ce-4f94-9d24-a3359639b3ff service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:00.173279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 845/1705] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:45:59 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:00.180925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:00.181721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:00.213423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:00.330779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 861/1706] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:00.338606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.338972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.339342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.378688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:00.381394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:00.412189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 846/1707] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:00.415433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:00.428799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f8c2-6fb3-4804-9954-ba2945eafd52 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:00.443641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:00.445799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.446278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.446563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.449934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:00.451311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:00.455574 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:00.504248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:00.532519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 862/1708] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:00.577262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2d3da-e191-467f-9676-4f6aae8b226d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:00.634923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:00.635975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.637981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.638710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.640554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:00.647989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:00.759579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 847/1709] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:00 2022] POST /identity/v3/projects => generated 365 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:00.766270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:00.783773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63060a12-a7e9-483a-95a7-e538a1b5b187 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:00.961367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:00.962628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:00.963026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:00.964630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:00.967481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:00.973728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:01.048661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 863/1710] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:00 2022] POST /identity/v3/users => generated 283 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:01.053925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:01.071092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab7d3a-0133-4b92-b757-681a5d5c9618 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:01.085648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:01.086210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:01.086787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:01.087278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.088037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:01.091270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:01.132057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 848/1711] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:01.132590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:01.152999 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb7cfb0-c66f-4b68-b2f4-eaa89e21094c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:01.164967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f59db-3599-41ed-945f-3bd736d797d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:01.166445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f59db-3599-41ed-945f-3bd736d797d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:01.167020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f59db-3599-41ed-945f-3bd736d797d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:01.167448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f59db-3599-41ed-945f-3bd736d797d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.168741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f59db-3599-41ed-945f-3bd736d797d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:01.173850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7f59db-3599-41ed-945f-3bd736d797d3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:01.213229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 864/1712] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:01.213352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:01.224730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680345e-acee-4c87-bab1-b61ea5d0cb6c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:01.238399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:01.240173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:01.241180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:01.242104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] PATH_INFO: `/v3/projects/16c0b954c05446fca0b4cfc687a4f6a4/users/1456bc2093b34f10a3a1b2f0d9bf6e2a/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.295991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c0b954c05446fca0b4cfc687a4f6a4, user_id=1456bc2093b34f10a3a1b2f0d9bf6e2a, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:01.299623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:01.331823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 849/1713] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:01 2022] PUT /identity/v3/projects/16c0b954c05446fca0b4cfc687a4f6a4/users/1456bc2093b34f10a3a1b2f0d9bf6e2a/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:01.342976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37214bc-6823-4293-a756-4afef21d77e4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:01.343521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37214bc-6823-4293-a756-4afef21d77e4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:01.343985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37214bc-6823-4293-a756-4afef21d77e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.406176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d37214bc-6823-4293-a756-4afef21d77e4 None None] MFA Rules not processed for user `1456bc2093b34f10a3a1b2f0d9bf6e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:01.407494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d37214bc-6823-4293-a756-4afef21d77e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:01.673101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 865/1714] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:01.703956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:01.706914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:01.902154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7040c1a6-60e2-4447-ad9e-0e01c6494e97 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:01.917058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd0657-bf75-4409-80ea-2989abdcf335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:01.917732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd0657-bf75-4409-80ea-2989abdcf335 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:01.918015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd0657-bf75-4409-80ea-2989abdcf335 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:01.918286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd0657-bf75-4409-80ea-2989abdcf335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:01.958301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd0657-bf75-4409-80ea-2989abdcf335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:01.963384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd0657-bf75-4409-80ea-2989abdcf335 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:01.997432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 850/1715] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:01 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:02.033702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:02.034436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:02.035508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:02.116210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] MFA Rules not processed for user `914bea2efd334e6c8dc31698dfabbf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:02.117620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:02.148262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:02.239417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 866/1716] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:02.251036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd64da-e320-4b3e-9724-4a5dcffcc569 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:02.251424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd64da-e320-4b3e-9724-4a5dcffcc569 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:02.251836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcd64da-e320-4b3e-9724-4a5dcffcc569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:02.285010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-8bcd64da-e320-4b3e-9724-4a5dcffcc569 None None] MFA Rules not processed for user `914bea2efd334e6c8dc31698dfabbf8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:02.287645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8bcd64da-e320-4b3e-9724-4a5dcffcc569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:02.302830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 851/1717] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:02.315161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:02.318291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:02.464772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34221d76-3f5a-42e7-bc00-6ccf13e592ec None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:02.471915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:02.473083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:02.473419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:02.473830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:02.485109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:02.486986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:02.499673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 867/1718] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:03.204234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab3de5-d29a-4dd6-81f9-bbbe541eb436 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:03.204762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab3de5-d29a-4dd6-81f9-bbbe541eb436 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:03.204762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab3de5-d29a-4dd6-81f9-bbbe541eb436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:03.275480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-52ab3de5-d29a-4dd6-81f9-bbbe541eb436 None None] MFA Rules not processed for user `c8b7cb87296a4ecea0b9ab44a0cbfd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:03.278087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-52ab3de5-d29a-4dd6-81f9-bbbe541eb436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:03.308215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:03.535610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 852/1719] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:03.546195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:03.572758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f51f8e-10f3-4d37-951e-4f0befedc460 service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:03.607030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002abfb-5c0a-4b8f-994e-9fc47782f5e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:03.608443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002abfb-5c0a-4b8f-994e-9fc47782f5e2 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:03.608722 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002abfb-5c0a-4b8f-994e-9fc47782f5e2 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:03.608994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002abfb-5c0a-4b8f-994e-9fc47782f5e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:03.637449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002abfb-5c0a-4b8f-994e-9fc47782f5e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:03.643183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002abfb-5c0a-4b8f-994e-9fc47782f5e2 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:03.668843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 868/1720] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:03.973900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a3eb5-edcd-47a0-9178-c13d86a903e0 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:03.974318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a3eb5-edcd-47a0-9178-c13d86a903e0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:03.974607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a3eb5-edcd-47a0-9178-c13d86a903e0 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:03.976288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 853/1721] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 05:46:03 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:03.984025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f12b89-c288-4ebc-83f3-df7498b3dd14 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:03.985039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f12b89-c288-4ebc-83f3-df7498b3dd14 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:03.985039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f12b89-c288-4ebc-83f3-df7498b3dd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:04.162114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c0f12b89-c288-4ebc-83f3-df7498b3dd14 None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:04.165211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c0f12b89-c288-4ebc-83f3-df7498b3dd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:04.422974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 869/1722] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:46:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 441 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:04.438146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a3eb5-edcd-47a0-9178-c13d86a903e0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:04.440582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7f8a3eb5-edcd-47a0-9178-c13d86a903e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:04.486813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a3eb5-edcd-47a0-9178-c13d86a903e0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:04.500314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1d0bd-43fd-4cf7-a25d-fc1e8542e8aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:04.501939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d0bd-43fd-4cf7-a25d-fc1e8542e8aa service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:04.501939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d0bd-43fd-4cf7-a25d-fc1e8542e8aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:04.501939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d0bd-43fd-4cf7-a25d-fc1e8542e8aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:04.512723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1d0bd-43fd-4cf7-a25d-fc1e8542e8aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:04.516341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1d0bd-43fd-4cf7-a25d-fc1e8542e8aa service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:04.536487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 854/1723] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:04.927099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345216-29f9-410d-8b72-245347af17a2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:04.927593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345216-29f9-410d-8b72-245347af17a2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:04.927709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d345216-29f9-410d-8b72-245347af17a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.013361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2d345216-29f9-410d-8b72-245347af17a2 None None] MFA Rules not processed for user `3d8467cc62d948038690af1fdf5bb96a`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:05.018884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2d345216-29f9-410d-8b72-245347af17a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.037484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:05.187523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 870/1724] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:05.197497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.197913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.198314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.240730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] MFA Rules not processed for user `3d8467cc62d948038690af1fdf5bb96a`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:05.242150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.248543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77df5a2-9f97-4677-82d2-ccc2a71fcd94 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.249341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77df5a2-9f97-4677-82d2-ccc2a71fcd94 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.249941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77df5a2-9f97-4677-82d2-ccc2a71fcd94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.262272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 855/1725] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:05.273082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:05.288093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f77df5a2-9f97-4677-82d2-ccc2a71fcd94 None None] MFA Rules not processed for user `0b18352a23444324b27d17d2cee636fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:05.289701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f77df5a2-9f97-4677-82d2-ccc2a71fcd94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:05.296459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f771-f1c2-4f87-b382-e0793f032617 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:05.314501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42eb1b3-0227-4368-99ca-7feee6e80028 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:05.315410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb1b3-0227-4368-99ca-7feee6e80028 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:05.330676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb1b3-0227-4368-99ca-7feee6e80028 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:05.331006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42eb1b3-0227-4368-99ca-7feee6e80028 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:05.359276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eb1b3-0227-4368-99ca-7feee6e80028 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:05.361736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42eb1b3-0227-4368-99ca-7feee6e80028 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:05.389972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 856/1726] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:05.451829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 871/1727] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:06.104599 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e966-0cf8-468a-aaba-be16145b81fb None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.105234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e966-0cf8-468a-aaba-be16145b81fb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.105525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43e966-0cf8-468a-aaba-be16145b81fb None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.107274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 857/1728] 10.222.0.73 () {54 vars in 912 bytes} [Wed Aug 24 05:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:06.113479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f88a4a-84a5-4b0d-bda4-2afb8281efc8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.113954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f88a4a-84a5-4b0d-bda4-2afb8281efc8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.114048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f88a4a-84a5-4b0d-bda4-2afb8281efc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.155621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-06f88a4a-84a5-4b0d-bda4-2afb8281efc8 None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:06.161732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-06f88a4a-84a5-4b0d-bda4-2afb8281efc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.182049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 872/1729] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:46:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:06.194001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43e966-0cf8-468a-aaba-be16145b81fb None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.195885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8e43e966-0cf8-468a-aaba-be16145b81fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.227508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43e966-0cf8-468a-aaba-be16145b81fb None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.233873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.234994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.236123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.236805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.248625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e9952-05da-48db-80a2-94a6cb9f807d None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.249119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.250336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e9952-05da-48db-80a2-94a6cb9f807d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.250616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e9952-05da-48db-80a2-94a6cb9f807d None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.251681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.252532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 873/1730] 10.222.0.73 () {54 vars in 961 bytes} [Wed Aug 24 05:46:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:06.259727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8849c-468c-4ad9-8caa-19b394537c6a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.260211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8849c-468c-4ad9-8caa-19b394537c6a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.260640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8849c-468c-4ad9-8caa-19b394537c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.292805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-00a8849c-468c-4ad9-8caa-19b394537c6a None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:06.294471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-00a8849c-468c-4ad9-8caa-19b394537c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.296118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 858/1731] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:06.395940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 874/1732] 10.222.0.73 () {58 vars in 1058 bytes} [Wed Aug 24 05:46:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 05:46:06.397052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:06.415469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e4e91a-b91c-4a2a-9c80-bdc2c318976e service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:06.424039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:06.424796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:06.425178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:06.425687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:06.427872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:06.447764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:06.450426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f934feb-05a3-45ee-998d-79f8dab07d18 service placement] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:06.468546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 859/1733] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:07.542872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:07.544130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:07.544570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:07.582715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] MFA Rules not processed for user `3d8467cc62d948038690af1fdf5bb96a`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:07.586178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:07.616691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 875/1734] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:08.022800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281cbe8-c3b7-4df7-bd8b-9e0a273214a1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:08.023371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281cbe8-c3b7-4df7-bd8b-9e0a273214a1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:08.023371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281cbe8-c3b7-4df7-bd8b-9e0a273214a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:08.076822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5281cbe8-c3b7-4df7-bd8b-9e0a273214a1 None None] MFA Rules not processed for user `c8b7cb87296a4ecea0b9ab44a0cbfd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:08.078253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5281cbe8-c3b7-4df7-bd8b-9e0a273214a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:08.107163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 860/1735] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:11.805247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:11.806911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:11.938506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:12.013130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3f15-c993-4079-9868-de22fcdf3091 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:12.036216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:12.036885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.037530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.038005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] PATH_INFO: `/v3/users/d0842c2ec2f94d6680ccacc39840dda1` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.039566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0842c2ec2f94d6680ccacc39840dda1)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:12.042116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:12.153299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0842c2ec2f94d6680ccacc39840dda1'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.155239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0842c2ec2f94d6680ccacc39840dda1'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.180993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0842c2ec2f94d6680ccacc39840dda1'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.182527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0842c2ec2f94d6680ccacc39840dda1'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.199721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0842c2ec2f94d6680ccacc39840dda1'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.220713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 876/1736] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:11 2022] DELETE /identity/v3/users/d0842c2ec2f94d6680ccacc39840dda1 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:12.225336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.225970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.226223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.251141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:12.252470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:12.324985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 861/1737] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:12.334782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:12.336333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:12.411212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48e720-3779-451f-aef3-9b22d8a95a18 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:12.418065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:12.418686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.419126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.422130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.433329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:12.434806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:12.452663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 877/1738] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:12.772345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:12.774018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:12.792205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035dd9aa-fbf3-4bc7-bc6c-48fb0cb8c4df None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:12.797092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:12.797811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.798094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.798363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] PATH_INFO: `/v3/projects/003b4ddd274d42d49fc1e5d70a772505` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.799514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=003b4ddd274d42d49fc1e5d70a772505)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:12.801427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:12.854933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:12.856804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '003b4ddd274d42d49fc1e5d70a772505', 'request_id': 'req-042e5da3-226f-4a03-a824-5c50e7bd19e4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.864757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '003b4ddd274d42d49fc1e5d70a772505', 'request_id': 'req-042e5da3-226f-4a03-a824-5c50e7bd19e4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.871670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '003b4ddd274d42d49fc1e5d70a772505', 'request_id': 'req-042e5da3-226f-4a03-a824-5c50e7bd19e4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.890937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] The token cache is being invalidate because project 003b4ddd274d42d49fc1e5d70a772505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:12.891232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.892016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:12.896835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 862/1739] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:12 2022] DELETE /identity/v3/projects/003b4ddd274d42d49fc1e5d70a772505 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:12.898090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:12.908671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690a9e6-3e60-400c-9fe3-3506f76b501c service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:12.975933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:12.976690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:12.977040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:12.977505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] PATH_INFO: `/v3/users/b2590e437303451aa93e7ab5d5336026` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:12.979408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2590e437303451aa93e7ab5d5336026)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:12.981463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:12.984212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:13.091037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2590e437303451aa93e7ab5d5336026'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.091750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2590e437303451aa93e7ab5d5336026'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.102099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2590e437303451aa93e7ab5d5336026'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.102804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2590e437303451aa93e7ab5d5336026'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.135144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6f338665-4da4-4fc3-821d-76cb4bde58f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2590e437303451aa93e7ab5d5336026'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.151465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 878/1740] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:12 2022] DELETE /identity/v3/users/b2590e437303451aa93e7ab5d5336026 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:13.559304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:13.561422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:13.575021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042e5da3-226f-4a03-a824-5c50e7bd19e4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:13.641843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:13.642627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:13.642977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:13.643259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] PATH_INFO: `/v3/projects/704b86c87a9e4642b736c4f99c33b18e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:13.644463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=704b86c87a9e4642b736c4f99c33b18e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:13.646531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:13.700467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704b86c87a9e4642b736c4f99c33b18e', 'request_id': 'req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.707218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704b86c87a9e4642b736c4f99c33b18e', 'request_id': 'req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.714498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704b86c87a9e4642b736c4f99c33b18e', 'request_id': 'req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.729907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] The token cache is being invalidate because project 704b86c87a9e4642b736c4f99c33b18e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:13.730197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.730850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8294e38c-a88d-41e5-b7fd-1d8e85d880c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:13.732912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 863/1741] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:13 2022] DELETE /identity/v3/projects/704b86c87a9e4642b736c4f99c33b18e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:13.737980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:13.738275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:13.738555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:13.761837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:13.763083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:13.829214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 879/1742] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:13.838270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:13.838556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:13.838831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:13.860362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:13.861601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:13.869313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 864/1743] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:13.874325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:13.883817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9290d92d-c9a3-48c1-b7f1-e9e7bb22ffb4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:13.888371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:13.889027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:13.889303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:13.889591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:13.890240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:13.890540 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:13.892134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:13.905294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 880/1744] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:13.909515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:13.919344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00847d7-fa2d-4ed8-9460-2640b4fb1587 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:13.924768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:13.925381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:13.925698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:13.926004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:13.926632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:13.928555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:13.963876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 865/1745] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:13 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:13.964783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:13.976958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4226ce-42d1-41b5-bcfc-b74dbc71891b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.040410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.040926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.041205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.041473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.042131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.044033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.055763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:14.075407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 881/1746] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:13 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:14.079505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.091185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03b6de-be78-4f9f-ac65-ea302e1f9f37 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.097808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.098476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.098703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.098974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.099637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.101269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.113027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 866/1747] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:14.116946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.134622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58c65f-91d0-49b9-ab8c-972e218c0684 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.166391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6a81c-625b-4f83-a0c8-d2a2da20c8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.183635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a81c-625b-4f83-a0c8-d2a2da20c8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.183635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a81c-625b-4f83-a0c8-d2a2da20c8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.183635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6a81c-625b-4f83-a0c8-d2a2da20c8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.183635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6a81c-625b-4f83-a0c8-d2a2da20c8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.183635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6a81c-625b-4f83-a0c8-d2a2da20c8f2 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.207613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 882/1748] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:14.208472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.234392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda9c6ba-317f-4f2b-91e9-16d764dcde02 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.239928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.240665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.241042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.241378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] PATH_INFO: `/v3/projects/5623bbb4cdfb44cbba4b3e677432ee8d/users/07f5bdb8940a44f78ebddd0a3f2a2856/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.261678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5623bbb4cdfb44cbba4b3e677432ee8d, user_id=07f5bdb8940a44f78ebddd0a3f2a2856, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.264620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.280311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 867/1749] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:14 2022] PUT /identity/v3/projects/5623bbb4cdfb44cbba4b3e677432ee8d/users/07f5bdb8940a44f78ebddd0a3f2a2856/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:14.290536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.290817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.291091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.319807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] MFA Rules not processed for user `07f5bdb8940a44f78ebddd0a3f2a2856`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:14.321053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:14.448034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 883/1750] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:14.452817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.457363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:14.530214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea173a2-7810-40aa-89b2-0bd69c2fffbc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.534802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.535412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.535689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.535958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.536591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.538508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.569426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 868/1751] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:14 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:14.573157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.586385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9f65c-36ce-4854-a1f4-3294cb41789a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.664669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.665941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.666593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.667282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.668836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.671872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.709276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 884/1752] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:14 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:14.713964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.736546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65adfbea-4288-4b7b-b1af-173087112711 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.743233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.744148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.744590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.745008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.745935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.748141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.769743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 869/1753] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:14.771927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.784101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d626d9-e2b6-4854-9304-561b9c9005ca admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.789680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676d7627-2da6-4f42-afc5-49bda2d83bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.790501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7627-2da6-4f42-afc5-49bda2d83bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.793908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7627-2da6-4f42-afc5-49bda2d83bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.794271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d7627-2da6-4f42-afc5-49bda2d83bd4 admin admin] PATH_INFO: `/v3/projects/95f371ea062c4f5ea0cedab52e687e7b/users/361fd409f54d41bea8ae44dc57eed714/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.816610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7627-2da6-4f42-afc5-49bda2d83bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f371ea062c4f5ea0cedab52e687e7b, user_id=361fd409f54d41bea8ae44dc57eed714, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.823273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676d7627-2da6-4f42-afc5-49bda2d83bd4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.839931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 885/1754] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:14 2022] PUT /identity/v3/projects/95f371ea062c4f5ea0cedab52e687e7b/users/361fd409f54d41bea8ae44dc57eed714/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:14.844203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:14.855234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47d8c2-b8cd-49e4-a751-a7c6d6f39acf admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:14.955300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:14.956120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.956481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.957012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.957618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:14.961164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:14.983810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:14.985152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:14.986398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:14.992447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 870/1755] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:14 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:15.024679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:15.051102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8bdf-89b1-4bf8-9416-70d5f11eb5bb admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:15.064541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af5ef3b-6f81-48d3-982a-f5a7473cf486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:15.066240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5ef3b-6f81-48d3-982a-f5a7473cf486 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:15.066240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5ef3b-6f81-48d3-982a-f5a7473cf486 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:15.066240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af5ef3b-6f81-48d3-982a-f5a7473cf486 admin admin] PATH_INFO: `/v3/projects/95f371ea062c4f5ea0cedab52e687e7b/users/361fd409f54d41bea8ae44dc57eed714/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:15.071999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5ef3b-6f81-48d3-982a-f5a7473cf486 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f371ea062c4f5ea0cedab52e687e7b, user_id=361fd409f54d41bea8ae44dc57eed714, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:15.073784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af5ef3b-6f81-48d3-982a-f5a7473cf486 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:15.104922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 871/1756] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:14 2022] PUT /identity/v3/projects/95f371ea062c4f5ea0cedab52e687e7b/users/361fd409f54d41bea8ae44dc57eed714/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:15.114165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:15.114592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:15.115130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:15.132282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:15.135015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:15.148739 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:15.199619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] MFA Rules not processed for user `361fd409f54d41bea8ae44dc57eed714`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:15.201233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:15.424355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:15.443274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 886/1757] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:46:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:15.508271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 872/1758] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:15.526581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:15.528309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:15.614626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a98fb80-50a4-4431-84df-8ea8e9c4795a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:15.620690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:15.620690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:15.620690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:15.621031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:15.634085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:15.636093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:15.676258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 887/1759] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:15.700430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:15.702217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:15.918484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1103a7-ce3b-462e-8903-01e3830a2c32 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:15.928601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:15.929241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:15.929714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:15.930016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:15.953212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:15.954984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:15.983922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 873/1760] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:46:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:46:16.011703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:16.027656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23ae31-5db3-497c-ae51-0056e4ba5754 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:16.057241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5545a526-8d04-4908-946c-abd2546e6a12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:16.058143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545a526-8d04-4908-946c-abd2546e6a12 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:16.063242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545a526-8d04-4908-946c-abd2546e6a12 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:16.064114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5545a526-8d04-4908-946c-abd2546e6a12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:16.189179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545a526-8d04-4908-946c-abd2546e6a12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:16.191015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5545a526-8d04-4908-946c-abd2546e6a12 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:16.224540 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 888/1761] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:17.612520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:17.616156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:17.845459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984adf48-71ad-45c4-9f08-6a1f13940a62 service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:17.867976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:17.868954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:17.869414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:17.869875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] PATH_INFO: `/v3/users/bc28b3b2a48e416ebbaa0a814baa2626` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:17.873275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc28b3b2a48e416ebbaa0a814baa2626)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:17.875899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:17.984156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc28b3b2a48e416ebbaa0a814baa2626'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.985708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc28b3b2a48e416ebbaa0a814baa2626'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.995951 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc28b3b2a48e416ebbaa0a814baa2626'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.996729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc28b3b2a48e416ebbaa0a814baa2626'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:17.999424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:18.010738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc28b3b2a48e416ebbaa0a814baa2626'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:18.042210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 874/1762] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:17 2022] DELETE /identity/v3/users/bc28b3b2a48e416ebbaa0a814baa2626 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:18.048690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.049268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.049691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.088157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:18.092629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:18.343163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:18.387958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 889/1763] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:18.444365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:18.451220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:18.680936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bace-624c-4268-8617-4e80f313dc60 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:18.688748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:18.689413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:18.689759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:18.690089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:18.702353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:18.710578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:18.729554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 875/1764] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:19.670196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:19.671486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:19.692253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187075f-8301-45a8-ba06-365d0827d219 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:19.701657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:19.702702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:19.703287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:19.704431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] PATH_INFO: `/v3/projects/4c9d6530be4346fe9bde87d62026017d` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:19.705910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9d6530be4346fe9bde87d62026017d)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:19.708137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:19.785800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9d6530be4346fe9bde87d62026017d', 'request_id': 'req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:19.794179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9d6530be4346fe9bde87d62026017d', 'request_id': 'req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:19.807534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9d6530be4346fe9bde87d62026017d', 'request_id': 'req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:19.821218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] The token cache is being invalidate because project 4c9d6530be4346fe9bde87d62026017d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:19.821519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:19.822898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:19.824481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 890/1765] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:19 2022] DELETE /identity/v3/projects/4c9d6530be4346fe9bde87d62026017d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:19.828755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:19.839618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436837a8-11a3-4c45-9668-fe418cf77cac service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:19.958822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:19.959882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:19.960331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:19.960751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] PATH_INFO: `/v3/users/ff5bbd6c562b44a9b25f32c9297b0e8d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:19.962308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5bbd6c562b44a9b25f32c9297b0e8d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:19.964436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:20.035626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5bbd6c562b44a9b25f32c9297b0e8d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.036360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5bbd6c562b44a9b25f32c9297b0e8d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.046829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5bbd6c562b44a9b25f32c9297b0e8d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.047975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5bbd6c562b44a9b25f32c9297b0e8d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.062021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9b309dd-b2f3-4b7c-8777-a9924afba88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5bbd6c562b44a9b25f32c9297b0e8d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.075015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:20.084591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 876/1766] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:19 2022] DELETE /identity/v3/users/ff5bbd6c562b44a9b25f32c9297b0e8d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:20.456728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:20.458358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:20.494566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee5b2b-089b-4d06-90cd-861edfcdbbd7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:20.639595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:20.640517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:20.641122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:20.641548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] PATH_INFO: `/v3/projects/99edf9fe9af64acbb4c6d1ed64797e50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:20.642904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99edf9fe9af64acbb4c6d1ed64797e50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:20.645184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:20.712264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99edf9fe9af64acbb4c6d1ed64797e50', 'request_id': 'req-972c561f-4330-4c3d-804b-de99d3f69910'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.719373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99edf9fe9af64acbb4c6d1ed64797e50', 'request_id': 'req-972c561f-4330-4c3d-804b-de99d3f69910'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.731228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99edf9fe9af64acbb4c6d1ed64797e50', 'request_id': 'req-972c561f-4330-4c3d-804b-de99d3f69910'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.744196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] The token cache is being invalidate because project 99edf9fe9af64acbb4c6d1ed64797e50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:20.744486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.745163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-972c561f-4330-4c3d-804b-de99d3f69910 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:20.746714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 891/1767] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:20 2022] DELETE /identity/v3/projects/99edf9fe9af64acbb4c6d1ed64797e50 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:20.751722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:20.752030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:20.752308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:20.775069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:20.776287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:20.854241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 877/1768] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:20.863799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:20.864112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:20.864361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:20.893017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:20.902633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:20.910438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:20.929442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 892/1769] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:20.933782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:20.946193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63fec0e-52ea-4d8a-8755-610605f578d8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:20.951829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:20.952520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:20.952873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:20.953249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:20.954102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:20.954523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:20.956391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:20.979378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 878/1770] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:20.981395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.002384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758cb9a3-0032-42da-af5c-cee2a62f926f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.015888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.016536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.016822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.017102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.017777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.019703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.063676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 893/1771] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:20 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:21.067868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.081792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c0bc20-6ae2-4b08-bc75-3a70bb2c003d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.159785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.160497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.160769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.161039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.161746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.163694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.188003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 879/1772] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:21 2022] POST /identity/v3/users => generated 285 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:21.192170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.213041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64e6b7f-c238-40d6-9ff9-241cbacf1e63 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.218580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.219198 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.219476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.219746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.220419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.222119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.235450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 894/1773] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:21.239980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.253304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4f23dd-837c-40dc-b178-2bcb1ff74b56 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.258681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20513897-9aca-4216-8a2e-b4069886f9ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.259306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20513897-9aca-4216-8a2e-b4069886f9ab admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.259592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20513897-9aca-4216-8a2e-b4069886f9ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.259863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20513897-9aca-4216-8a2e-b4069886f9ab admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.260512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20513897-9aca-4216-8a2e-b4069886f9ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.262161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20513897-9aca-4216-8a2e-b4069886f9ab admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.275675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 880/1774] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:21.280427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.291726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4586f5c1-c092-4d2a-b6d8-4a1fbb810553 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.300881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.301906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.302341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.302695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] PATH_INFO: `/v3/projects/0f7346df7047474b8437236ff2bae4b6/users/ad551c4c4be542098ee59899c1885cd4/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.324927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7346df7047474b8437236ff2bae4b6, user_id=ad551c4c4be542098ee59899c1885cd4, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.328779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.348012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 895/1775] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:21 2022] PUT /identity/v3/projects/0f7346df7047474b8437236ff2bae4b6/users/ad551c4c4be542098ee59899c1885cd4/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:21.359721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.360011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.360313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.391707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] MFA Rules not processed for user `ad551c4c4be542098ee59899c1885cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:21.393067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:21.454937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:21.476289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 881/1776] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:21 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:21.480969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.573251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8692b9da-20e1-46a7-aa08-7f2d399a4b61 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.578815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.579505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.579791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.580073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.580721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.583015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.646048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 896/1777] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:21 2022] POST /identity/v3/projects => generated 367 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:21.648116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.661652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ef6b8-042c-4c5c-8e97-e31c53217572 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.778216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.779076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.779564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.779891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.780570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.782751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.812393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 882/1778] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:21 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:21.829050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.850170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2b06d-f30c-4e46-9890-84ee9f7fe9ce admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.858475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.859341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.859789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.860167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.861252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.863645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.897019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 897/1779] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:21.901425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:21.919204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fc834-b2d2-465c-a3e5-7538920c24bc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:21.926522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:21.928321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:21.928713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:21.929092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] PATH_INFO: `/v3/projects/23ec15b4502842f69fc997b0b2ec2f7e/users/bb1d9fdb45e34e7b839f71e079dbc7f5/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:21.967713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ec15b4502842f69fc997b0b2ec2f7e, user_id=bb1d9fdb45e34e7b839f71e079dbc7f5, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:21.971194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:21.989605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 883/1780] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:21 2022] PUT /identity/v3/projects/23ec15b4502842f69fc997b0b2ec2f7e/users/bb1d9fdb45e34e7b839f71e079dbc7f5/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:21.993290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:22.006699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d48d9-1782-4fa5-af87-d0f89a7df793 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:22.088172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:22.095358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:22.102272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:22.103040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.103404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.103772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.104812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:22.106629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:22.116667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a317915a-e73d-4d08-8a0c-6362d8214b01 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:22.123249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:22.124203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.124519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.124795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] PATH_INFO: `/v3/users/07f5bdb8940a44f78ebddd0a3f2a2856` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.126016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07f5bdb8940a44f78ebddd0a3f2a2856)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:22.126535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 898/1781] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:21 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:22.128076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:22.134163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:22.161114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57486280-b41e-424e-ae88-c2b253cf9aaa admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:22.166913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db530fa1-55f6-4e8d-a2d5-3fb55965edcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:22.177458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db530fa1-55f6-4e8d-a2d5-3fb55965edcb admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.177458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db530fa1-55f6-4e8d-a2d5-3fb55965edcb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.177458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db530fa1-55f6-4e8d-a2d5-3fb55965edcb admin admin] PATH_INFO: `/v3/projects/23ec15b4502842f69fc997b0b2ec2f7e/users/bb1d9fdb45e34e7b839f71e079dbc7f5/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.179738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db530fa1-55f6-4e8d-a2d5-3fb55965edcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=23ec15b4502842f69fc997b0b2ec2f7e, user_id=bb1d9fdb45e34e7b839f71e079dbc7f5, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:22.182813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db530fa1-55f6-4e8d-a2d5-3fb55965edcb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:22.201893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:22.219282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 899/1782] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:22 2022] PUT /identity/v3/projects/23ec15b4502842f69fc997b0b2ec2f7e/users/bb1d9fdb45e34e7b839f71e079dbc7f5/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:22.231672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.232501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.233442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.237973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07f5bdb8940a44f78ebddd0a3f2a2856'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:22.238918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07f5bdb8940a44f78ebddd0a3f2a2856'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:22.249361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07f5bdb8940a44f78ebddd0a3f2a2856'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:22.250866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07f5bdb8940a44f78ebddd0a3f2a2856'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:22.283076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] MFA Rules not processed for user `bb1d9fdb45e34e7b839f71e079dbc7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:22.284637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:22.286495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3853fdfc-515e-4e1d-a79b-b5a80d111013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07f5bdb8940a44f78ebddd0a3f2a2856'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:22.323964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 884/1783] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:22 2022] DELETE /identity/v3/users/07f5bdb8940a44f78ebddd0a3f2a2856 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:22.328204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.328498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.328775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.348235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:22.349517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:22.372504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 900/1784] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:22.378760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:22.389034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:22.390677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:22.437518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 885/1785] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:22.447596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:22.449053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:22.455683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb3e77-a466-4dbc-9200-ed588b4486fd None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:22.521274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:22.521911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.522161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985be427-c0ef-42bb-a251-7d016d1d9507 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:22.522305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.522528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.527309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:22.528448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:22.528732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:22.529087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:22.541321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:22.543106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:22.555051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 886/1786] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:22.600062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:22.613041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:22.614795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:22.637362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 901/1787] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:23.693342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:23.694844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:23.717219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206376b-44e2-444f-81e1-7593e4334368 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:23.728912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:23.731721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:23.732872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:23.735833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] PATH_INFO: `/v3/projects/5623bbb4cdfb44cbba4b3e677432ee8d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:23.736864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5623bbb4cdfb44cbba4b3e677432ee8d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:23.741278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:23.860714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5623bbb4cdfb44cbba4b3e677432ee8d', 'request_id': 'req-b07c9e27-696b-408c-b554-e190063113bc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:23.868653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5623bbb4cdfb44cbba4b3e677432ee8d', 'request_id': 'req-b07c9e27-696b-408c-b554-e190063113bc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:23.876691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5623bbb4cdfb44cbba4b3e677432ee8d', 'request_id': 'req-b07c9e27-696b-408c-b554-e190063113bc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:23.912593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] The token cache is being invalidate because project 5623bbb4cdfb44cbba4b3e677432ee8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:23.913207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:23.914084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:23.924889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 887/1788] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:23 2022] DELETE /identity/v3/projects/5623bbb4cdfb44cbba4b3e677432ee8d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:23.929467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:23.944531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5aee96-ed72-4883-b2f2-4010372c8f7d service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:24.050602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:24.051295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:24.052064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:24.052064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] PATH_INFO: `/v3/users/361fd409f54d41bea8ae44dc57eed714` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:24.054148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361fd409f54d41bea8ae44dc57eed714)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:24.056761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:24.137315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361fd409f54d41bea8ae44dc57eed714'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:24.138033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361fd409f54d41bea8ae44dc57eed714'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:24.155714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361fd409f54d41bea8ae44dc57eed714'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:24.156569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361fd409f54d41bea8ae44dc57eed714'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:24.179413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361fd409f54d41bea8ae44dc57eed714'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:24.221115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 902/1789] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:23 2022] DELETE /identity/v3/users/361fd409f54d41bea8ae44dc57eed714 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:24.737275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:24.741577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:24.769875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c9e27-696b-408c-b554-e190063113bc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:24.847557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:24.864520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:24.865928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:24.866763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:24.867504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] PATH_INFO: `/v3/users/1456bc2093b34f10a3a1b2f0d9bf6e2a` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:24.870444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=1456bc2093b34f10a3a1b2f0d9bf6e2a)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:24.873298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:24.874820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:24.877063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:24.905228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c252-f2c4-472e-8ddd-707ca3a630b6 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:24.933596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:24.936304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:24.938542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:24.938948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] PATH_INFO: `/v3/projects/95f371ea062c4f5ea0cedab52e687e7b` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:24.942007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f371ea062c4f5ea0cedab52e687e7b)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:24.947674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:25.022218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f371ea062c4f5ea0cedab52e687e7b', 'request_id': 'req-6503e0b1-975f-401e-90cb-6ad123d29f2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.030053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f371ea062c4f5ea0cedab52e687e7b', 'request_id': 'req-6503e0b1-975f-401e-90cb-6ad123d29f2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.041147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1456bc2093b34f10a3a1b2f0d9bf6e2a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.042640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1456bc2093b34f10a3a1b2f0d9bf6e2a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.045426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f371ea062c4f5ea0cedab52e687e7b', 'request_id': 'req-6503e0b1-975f-401e-90cb-6ad123d29f2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.048981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:25.058938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1456bc2093b34f10a3a1b2f0d9bf6e2a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.059683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1456bc2093b34f10a3a1b2f0d9bf6e2a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.078577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-18de2fb2-8981-4ea7-a91a-b2e52efc32ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1456bc2093b34f10a3a1b2f0d9bf6e2a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.091440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] The token cache is being invalidate because project 95f371ea062c4f5ea0cedab52e687e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:25.095266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.097074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6503e0b1-975f-401e-90cb-6ad123d29f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.099982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 903/1790] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:24 2022] DELETE /identity/v3/projects/95f371ea062c4f5ea0cedab52e687e7b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:25.104903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1707-d329-4a42-a4c2-991be838b450 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.105457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1707-d329-4a42-a4c2-991be838b450 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.107266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5f1707-d329-4a42-a4c2-991be838b450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.142336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 888/1791] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:24 2022] DELETE /identity/v3/users/1456bc2093b34f10a3a1b2f0d9bf6e2a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:25.157954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.157954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.157954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.170372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9f5f1707-d329-4a42-a4c2-991be838b450 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:25.172654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9f5f1707-d329-4a42-a4c2-991be838b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.202176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:25.204572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.294568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 904/1792] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:25.303787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.305105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.305105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.336848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 889/1793] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:25.344121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.345556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.351682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:25.353157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.365875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33873d5-54fe-4403-a842-0741b59d07a5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:25.367243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 905/1794] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:25.368393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.369873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.371149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:25.371932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.372223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.372500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] PATH_INFO: `/v3/users/ad551c4c4be542098ee59899c1885cd4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.381974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad551c4c4be542098ee59899c1885cd4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:25.385084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:25.468674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed1a7d5-e18d-4f00-9ce3-c2af81a3545f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:25.476964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:25.477961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.478387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.478972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.501222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad551c4c4be542098ee59899c1885cd4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.501664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:25.503285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:25.506037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad551c4c4be542098ee59899c1885cd4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.519631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad551c4c4be542098ee59899c1885cd4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.520502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad551c4c4be542098ee59899c1885cd4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.527302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 906/1795] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:25.537302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.543924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-19f6ecd4-68c6-4347-944d-e8fad3a248d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad551c4c4be542098ee59899c1885cd4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:25.552963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a395e7-54c0-40b7-99b2-54ea99e97074 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:25.566344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:25.572332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:25.573924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.574847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.577247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.579280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:25.580469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:25.586687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:25.597613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 890/1796] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:25 2022] DELETE /identity/v3/users/ad551c4c4be542098ee59899c1885cd4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:25.602504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.603216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.603633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.625779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 907/1797] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:25.628419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.632401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.648836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe66a51-6a28-4cef-b5c8-806028a98a48 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:25.668138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:25.673128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.747185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:25.747185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:25.747185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:25.747185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:25.747624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:25.747624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:25.795026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 891/1798] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:25.807170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.813625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:25.843407 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 908/1799] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/projects => generated 409 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:25.848009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:25.855171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:25.922248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6d5f1-d40a-45d9-b484-1cf1d22ad3c6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.030704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.035827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.036365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.036646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.052179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41621521-9f3d-44ca-bace-0a13c03e6e06 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.065294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.068894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.070303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.071776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.072798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.073150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.075112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.079582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.097669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 892/1800] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:26.118428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 909/1801] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:25 2022] POST /identity/v3/users => generated 305 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:26.124268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.146182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06737fb-47f7-40fa-b71c-db8963a07fde service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.151712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.152336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.153229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.153229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.154411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.156445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.171604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 893/1802] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:26.175523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.188294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b3efd-b168-4dc4-a22b-80f7e5e873aa admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.198204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.199665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.200840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.202178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.203913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.209968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.228829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 910/1803] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:26.233711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.253634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c608d9c-83b1-4363-a608-22d11db1a4a9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.260814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5b9ed-e45d-4968-b7f1-0a24c686c3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.262201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5b9ed-e45d-4968-b7f1-0a24c686c3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.262737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5b9ed-e45d-4968-b7f1-0a24c686c3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.266983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5b9ed-e45d-4968-b7f1-0a24c686c3d0 admin admin] PATH_INFO: `/v3/projects/c6b1689d55c24d10b48e42a56fafba12/users/9a71ceb0c6a54543beaa7ba7a94375d3/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.304250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5b9ed-e45d-4968-b7f1-0a24c686c3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b1689d55c24d10b48e42a56fafba12, user_id=9a71ceb0c6a54543beaa7ba7a94375d3, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.322397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.324857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.329177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5b9ed-e45d-4968-b7f1-0a24c686c3d0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.355310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91caed8-11a7-49ca-807b-6fdc63dfcdf6 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.382633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 894/1804] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:26 2022] PUT /identity/v3/projects/c6b1689d55c24d10b48e42a56fafba12/users/9a71ceb0c6a54543beaa7ba7a94375d3/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:26.403271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.403870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.404329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.447518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:26.469595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] MFA Rules not processed for user `9a71ceb0c6a54543beaa7ba7a94375d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:26.471302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.587594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.588415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.588867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.589150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] PATH_INFO: `/v3/users/d89b82ec277f4139a80724a975d74f22` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.590616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d89b82ec277f4139a80724a975d74f22)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.593158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.599971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 895/1805] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:26 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:26.600956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.606476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.633375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934def31-bcfd-40c4-9493-0dc256ca5a1c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.643478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.644289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.644782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.645025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] PATH_INFO: `/v3/projects/16c0b954c05446fca0b4cfc687a4f6a4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.653427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c0b954c05446fca0b4cfc687a4f6a4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.656727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.683600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd89b82ec277f4139a80724a975d74f22'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.684468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd89b82ec277f4139a80724a975d74f22'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.695193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd89b82ec277f4139a80724a975d74f22'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.695872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd89b82ec277f4139a80724a975d74f22'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.713325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd89b82ec277f4139a80724a975d74f22'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.725876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:26.735706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 911/1806] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:26 2022] DELETE /identity/v3/users/d89b82ec277f4139a80724a975d74f22 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:26.736890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.738917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c0b954c05446fca0b4cfc687a4f6a4', 'request_id': 'req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.744244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.754490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c0b954c05446fca0b4cfc687a4f6a4', 'request_id': 'req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.775749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c0b954c05446fca0b4cfc687a4f6a4', 'request_id': 'req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.805353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] The token cache is being invalidate because project 16c0b954c05446fca0b4cfc687a4f6a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:26.811385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.814469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:26.819588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 896/1807] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:26 2022] DELETE /identity/v3/projects/16c0b954c05446fca0b4cfc687a4f6a4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:26.820849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:26.822015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:26.902551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93d3dc2-5d1f-4f16-a793-e3fea0729b55 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.908323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96f681d-33b2-4619-a78b-45afb92d5ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:26.909066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f681d-33b2-4619-a78b-45afb92d5ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.909316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f681d-33b2-4619-a78b-45afb92d5ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.909672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96f681d-33b2-4619-a78b-45afb92d5ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.910312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f681d-33b2-4619-a78b-45afb92d5ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:26.916620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96f681d-33b2-4619-a78b-45afb92d5ca8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:26.956546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 912/1808] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:26 2022] POST /identity/v3/projects => generated 407 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:26.957896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadf05c-0277-46d5-a2fd-325f4d4e6347 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:26.958174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadf05c-0277-46d5-a2fd-325f4d4e6347 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:26.958447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadf05c-0277-46d5-a2fd-325f4d4e6347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:26.963988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f5eb1a-63e8-45ea-87d3-83bc4927e5ab admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:26.984766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4fadf05c-0277-46d5-a2fd-325f4d4e6347 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:26.986836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4fadf05c-0277-46d5-a2fd-325f4d4e6347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.015207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:27.071928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.072583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.072865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.073135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] PATH_INFO: `/v3/projects/0f7346df7047474b8437236ff2bae4b6` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.074336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7346df7047474b8437236ff2bae4b6)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.078552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.103682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 913/1809] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:27.116416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.116807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.117213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.166897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:27.166897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.171645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7346df7047474b8437236ff2bae4b6', 'request_id': 'req-87caa168-4c09-4bf4-b9f2-69524be77e44'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.179889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7346df7047474b8437236ff2bae4b6', 'request_id': 'req-87caa168-4c09-4bf4-b9f2-69524be77e44'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.187059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7346df7047474b8437236ff2bae4b6', 'request_id': 'req-87caa168-4c09-4bf4-b9f2-69524be77e44'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.207778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] The token cache is being invalidate because project 0f7346df7047474b8437236ff2bae4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:27.208092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.208756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.211090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 897/1810] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:26 2022] DELETE /identity/v3/projects/0f7346df7047474b8437236ff2bae4b6 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:27.215770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.218197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.294165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 914/1811] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:27.303510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.305412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.310445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caa168-4c09-4bf4-b9f2-69524be77e44 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.323627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.324463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.324836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.325243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.327049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.330621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.365372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:27.389635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 898/1812] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:27 2022] POST /identity/v3/users => generated 310 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:27.390684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.392655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.413809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571cd156-12e7-45a4-972b-f83e4a150838 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.418809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.419899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.420190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.420461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] PATH_INFO: `/v3/users/bb1d9fdb45e34e7b839f71e079dbc7f5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.421685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1d9fdb45e34e7b839f71e079dbc7f5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.423942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.443431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42bc88c-a06a-4610-8af5-2f03872882c1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.447778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.448395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.448665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.448935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.450917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.465305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.467326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a321f0-1503-4d0c-8e5e-a0b1855bfd80 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.479802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 915/1813] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:27.480271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.480632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.480970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.487033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:27.489072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1d9fdb45e34e7b839f71e079dbc7f5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.489999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1d9fdb45e34e7b839f71e079dbc7f5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.503632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1d9fdb45e34e7b839f71e079dbc7f5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.509588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1d9fdb45e34e7b839f71e079dbc7f5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.511112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:27.514044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.522874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1d9fdb45e34e7b839f71e079dbc7f5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:27.538064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 916/1814] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:27.538064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.538064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:27.554367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 899/1815] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:27 2022] DELETE /identity/v3/users/bb1d9fdb45e34e7b839f71e079dbc7f5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:27.555004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.566606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62638458-30fc-4713-b6e4-73993a54d1d2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.651174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee585735-d8b9-48a7-add3-ba1c7e408c65 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.671736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.672293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.672599 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.672876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.673540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:27.673871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.675642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.693273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 900/1816] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:27.697162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.708491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.709530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.709668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d65824-380d-4104-8a0c-7caf9058460b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.709898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.710188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.711276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.712971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.715098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.716411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.716411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.716411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.717159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.719472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.726951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 917/1817] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:27.732464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.746833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8cdc8-030c-4734-ad5b-f9aeb3a3adfb admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.754714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.755398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.755735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.756016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] PATH_INFO: `/v3/projects/75aad4108f0d45d6a8da5e35103ef348/users/ec4376b2d18f4a8c80f689b0468d67a6/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.759912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 901/1818] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:27 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:27.760956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.801902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75aad4108f0d45d6a8da5e35103ef348, user_id=ec4376b2d18f4a8c80f689b0468d67a6, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.807241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.838021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 918/1819] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:27 2022] PUT /identity/v3/projects/75aad4108f0d45d6a8da5e35103ef348/users/ec4376b2d18f4a8c80f689b0468d67a6/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:27.839803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:27.850127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21453817-f05f-4ae7-acec-2d5b18ecb25d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.925484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:27.967641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.968360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.968714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.968938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253072e3-9ad6-4656-aeff-b31bcb0fb3d5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:27.969376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.970189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.972156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:27.975005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:27.975636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:27.975918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:27.976864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:27.977839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:27.979526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.003757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 919/1820] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:27 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:28.004808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.006926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.024268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6440b03d-2bf4-4411-8104-94917adec2e7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.028231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 902/1821] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:27 2022] POST /identity/v3/users => generated 290 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:28.028231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.029903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.030622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.031004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.031794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] PATH_INFO: `/v3/projects/e35cf25979d449219f522afe907f1fb8` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.032814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35cf25979d449219f522afe907f1fb8)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.035942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.038944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15dfde-af79-4d0d-a865-0b617f574478 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.045048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.045863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.046154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.046438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] PATH_INFO: `/v3/projects/75aad4108f0d45d6a8da5e35103ef348/users/ec4376b2d18f4a8c80f689b0468d67a6/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.048431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=75aad4108f0d45d6a8da5e35103ef348, user_id=ec4376b2d18f4a8c80f689b0468d67a6, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.051418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.065192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 903/1822] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:28 2022] PUT /identity/v3/projects/75aad4108f0d45d6a8da5e35103ef348/users/ec4376b2d18f4a8c80f689b0468d67a6/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:28.066357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.073443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:28.097670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35cf25979d449219f522afe907f1fb8', 'request_id': 'req-ab819acf-a44f-4d5e-972f-6d0b8611c564'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.105025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35cf25979d449219f522afe907f1fb8', 'request_id': 'req-ab819acf-a44f-4d5e-972f-6d0b8611c564'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.118048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35cf25979d449219f522afe907f1fb8', 'request_id': 'req-ab819acf-a44f-4d5e-972f-6d0b8611c564'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.132966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] The token cache is being invalidate because project e35cf25979d449219f522afe907f1fb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:28.133262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.133984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.135698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 920/1823] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:28 2022] DELETE /identity/v3/projects/e35cf25979d449219f522afe907f1fb8 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:28.136481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.140993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.189079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605a82b-de7c-479d-9e06-df87f5baae2c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.213098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819acf-a44f-4d5e-972f-6d0b8611c564 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.219180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.219909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.220197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.221212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] PATH_INFO: `/v3/projects/23ec15b4502842f69fc997b0b2ec2f7e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.223588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=23ec15b4502842f69fc997b0b2ec2f7e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.226869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.266459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f3451-9873-443f-8297-1610008d4ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.267212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3451-9873-443f-8297-1610008d4ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.267646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3451-9873-443f-8297-1610008d4ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.268031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f3451-9873-443f-8297-1610008d4ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.269182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f3451-9873-443f-8297-1610008d4ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.270952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f3451-9873-443f-8297-1610008d4ea5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.283230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 904/1824] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:28.284500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.284867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.285173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.291258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23ec15b4502842f69fc997b0b2ec2f7e', 'request_id': 'req-99cea639-640d-42af-85cc-fbc0e35a3cab'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.297354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23ec15b4502842f69fc997b0b2ec2f7e', 'request_id': 'req-99cea639-640d-42af-85cc-fbc0e35a3cab'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.305801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23ec15b4502842f69fc997b0b2ec2f7e', 'request_id': 'req-99cea639-640d-42af-85cc-fbc0e35a3cab'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.316021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] MFA Rules not processed for user `ec4376b2d18f4a8c80f689b0468d67a6`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:28.317318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.318464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] The token cache is being invalidate because project 23ec15b4502842f69fc997b0b2ec2f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:28.318840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.319416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.320931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 921/1825] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:28 2022] DELETE /identity/v3/projects/23ec15b4502842f69fc997b0b2ec2f7e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:28.321730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.323039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.372295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:28.383797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:28.410749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 905/1826] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:28 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:28.411943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.413521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.423614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cea639-640d-42af-85cc-fbc0e35a3cab admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.428141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfabc4c0-e79b-4c24-bdc0-d0f326aa2c53 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.429043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.429892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.430116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.430397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] PATH_INFO: `/v3/users/c9310661b957439fad664e1b5a0da8ac` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.431730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9310661b957439fad664e1b5a0da8ac)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.433540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689142fc-a7c5-4e01-860d-d4a835c85fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.434175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689142fc-a7c5-4e01-860d-d4a835c85fca admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.434468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689142fc-a7c5-4e01-860d-d4a835c85fca admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.434589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.435912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689142fc-a7c5-4e01-860d-d4a835c85fca admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.436127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689142fc-a7c5-4e01-860d-d4a835c85fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.438125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689142fc-a7c5-4e01-860d-d4a835c85fca admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.454916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 906/1827] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:28.456495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.456881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.457254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.479924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:28.481299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.489580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9310661b957439fad664e1b5a0da8ac'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.490452 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9310661b957439fad664e1b5a0da8ac'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.499323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9310661b957439fad664e1b5a0da8ac'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.500214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9310661b957439fad664e1b5a0da8ac'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.517765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-def7824f-c4b4-449f-b097-578d63e41cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9310661b957439fad664e1b5a0da8ac'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:28.531870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 922/1828] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:28 2022] DELETE /identity/v3/users/c9310661b957439fad664e1b5a0da8ac => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:28.533121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.533408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.533825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.541972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 907/1829] 10.222.0.57 () {58 vars in 1015 bytes} [Wed Aug 24 05:46:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:28.543082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.544579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.553586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:28.554916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.610605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907b4bc-910e-4fdd-afa3-99de90d56cef None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.633799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 923/1830] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:28.633799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.635120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.648414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1857b-db3e-414a-a13c-ad8c7af32140 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.653911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.654496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.654949 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.655224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] PATH_INFO: `/v3/projects/7a7e4d68dee448609fd7e2f8d6c55694/users/f03b6ec092d041ada29b0b92cc70dc5a/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.681679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7e4d68dee448609fd7e2f8d6c55694, user_id=f03b6ec092d041ada29b0b92cc70dc5a, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.681952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.682610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.682898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.684238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.684384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.688363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.705550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 924/1831] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:28 2022] PUT /identity/v3/projects/7a7e4d68dee448609fd7e2f8d6c55694/users/f03b6ec092d041ada29b0b92cc70dc5a/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:28.706531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.709016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.762176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:28.783926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53c86d-42f4-4c1f-a8ab-9fc45bd10db3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.788957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.789799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.790191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.790571 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.793032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:28.800734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.802552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.814926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 925/1832] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:28.815722 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.817597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:28.819007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.821195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.833343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 908/1833] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:28.834341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.836018 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6017b5d-249e-4473-b231-9556b0142d6a service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.842945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.843769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.844204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.844991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.856439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.860845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.875957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 926/1834] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:28.877222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.891480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc914e7-2054-4348-8b4c-320643df3c8e service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.924074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.924710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.924980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.925251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.945634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63807d4e-434c-4d5e-a31e-7b1f0adb34c1 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.952423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48334f49-5372-4a27-b2ef-1982cbc3dfca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.958339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.958459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48334f49-5372-4a27-b2ef-1982cbc3dfca service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.958459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48334f49-5372-4a27-b2ef-1982cbc3dfca service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.958459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48334f49-5372-4a27-b2ef-1982cbc3dfca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:28.958671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.963766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 927/1835] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:28.964476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:28.974720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8905504-ccac-443b-bdf2-51073a76972a service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:28.975649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48334f49-5372-4a27-b2ef-1982cbc3dfca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:28.978056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48334f49-5372-4a27-b2ef-1982cbc3dfca service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:28.980463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8461c169-4b70-4aeb-a110-efce7a7066d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:28.981195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461c169-4b70-4aeb-a110-efce7a7066d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:28.981548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461c169-4b70-4aeb-a110-efce7a7066d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:28.982887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8461c169-4b70-4aeb-a110-efce7a7066d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.005684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461c169-4b70-4aeb-a110-efce7a7066d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.007325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8461c169-4b70-4aeb-a110-efce7a7066d6 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.007984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 909/1836] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:29.010679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.012147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.012451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.016993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 928/1837] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:29.018379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.018757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.019100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.053483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] MFA Rules not processed for user `f03b6ec092d041ada29b0b92cc70dc5a`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:29.055045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.057091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:29.058411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.135816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 929/1838] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:29.136685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.138063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.209336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 910/1839] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:29.210497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.218155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84bf45-2a14-4866-928c-59060b5e266a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.224846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.225506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.225830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.226073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] PATH_INFO: `/v3/projects/51e0ae07d5c84e29b85bbb01f0c7c030` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.227623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e0ae07d5c84e29b85bbb01f0c7c030)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.228121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62e0e1e-049c-42a9-a8a1-90c662d80a44 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.229801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.236476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.237358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.237845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.239071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.239867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.243582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.298035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:29.306906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 911/1840] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:29.310654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.320780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e0ae07d5c84e29b85bbb01f0c7c030', 'request_id': 'req-62d2084b-0b81-4a22-b18d-46e86cfb9391'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:29.338616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e0ae07d5c84e29b85bbb01f0c7c030', 'request_id': 'req-62d2084b-0b81-4a22-b18d-46e86cfb9391'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:29.359598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e0ae07d5c84e29b85bbb01f0c7c030', 'request_id': 'req-62d2084b-0b81-4a22-b18d-46e86cfb9391'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:29.373729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] The token cache is being invalidate because project 51e0ae07d5c84e29b85bbb01f0c7c030 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:29.374022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:29.374722 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:29.379416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 930/1841] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:29 2022] DELETE /identity/v3/projects/51e0ae07d5c84e29b85bbb01f0c7c030 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:29.379416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.379416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.400994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754fb1b7-b846-4da3-afc5-8e75de49fca0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.406733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.407493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.407878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.408256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.411499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.449076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:29.453699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d2084b-0b81-4a22-b18d-46e86cfb9391 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.506909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.509159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.523936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 912/1842] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:29 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:29.526399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.527960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.541064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.541748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.542037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.545763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdacbc6-bd5b-4e0f-addc-32d5f303a3d9 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.547639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.548373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:29.548725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.550357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.551845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbfddf-10ea-481d-be78-67f17413f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.552740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbfddf-10ea-481d-be78-67f17413f9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.553169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbfddf-10ea-481d-be78-67f17413f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.553624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbfddf-10ea-481d-be78-67f17413f9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.554404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbfddf-10ea-481d-be78-67f17413f9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.556544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdbfddf-10ea-481d-be78-67f17413f9c2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.587780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 913/1843] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/users => generated 296 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:29.587780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.587780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.587780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.595241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 931/1844] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:29.599780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.611948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:29.613263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.633589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 914/1845] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:29.634605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.634728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5c4fe-76b5-4afb-88f1-d235fccfde72 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.646128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4451e0-cc8f-4b47-b504-369e451444c0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.650032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38d5f7e-515c-457d-ad2b-8d018dececef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.650660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d5f7e-515c-457d-ad2b-8d018dececef admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.650861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.650982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d5f7e-515c-457d-ad2b-8d018dececef admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.651199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38d5f7e-515c-457d-ad2b-8d018dececef admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.651482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.651766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.651872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d5f7e-515c-457d-ad2b-8d018dececef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.652040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.652673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.656361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.664167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38d5f7e-515c-457d-ad2b-8d018dececef admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.690631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 932/1846] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:29.690631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.690631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.690631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.703780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 915/1847] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:29.703780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.712954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:29.714225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:29.796068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:29.814106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 933/1848] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:29.815183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.820382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9195ee91-e63c-40cd-85c1-8635975e4b9a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.827790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9c9ed-3362-4b9b-a6eb-3bcd62de0b5c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.828536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.829522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.830042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.830451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] PATH_INFO: `/v3/projects/cc753a070be64b3c807f848146433c14/users/29069b6f245b465d9a93ba3dc9e86247/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.835869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:29.836734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:29.836734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:29.837154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:29.837931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.840154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.853180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc753a070be64b3c807f848146433c14, user_id=29069b6f245b465d9a93ba3dc9e86247, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:29.857884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:29.875752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 934/1849] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:29 2022] POST /identity/v3/users => generated 293 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:29.877531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:29.897075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c0b47-51f9-4317-b545-7c4317c8eec4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:29.898469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 916/1850] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:29 2022] PUT /identity/v3/projects/cc753a070be64b3c807f848146433c14/users/29069b6f245b465d9a93ba3dc9e86247/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:29.904496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.006890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fecfd-28f4-4c9e-8721-c4502c213094 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.012948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.013642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.013924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.014917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.014917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.016637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.027938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:30.031860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 917/1851] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:29 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:30.032860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.044050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c70cd4-e4f4-4aa8-ab1b-6e99a8733133 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.046736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.048705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.049001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.049813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.050418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.050695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.050804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.050959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.051460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:30.051577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.051778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.053254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.055377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.069192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 918/1852] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:30.070312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.082422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f92026-2ab9-40e6-8099-fcedef306a26 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.087479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.088084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.088364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.088711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.089596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.092406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.095348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 935/1853] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:30.096317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.104628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 919/1854] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:30.109594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.120147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a3343-2e66-4821-823f-be15efc29fba admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.123519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c5aeef-3655-43c4-9c23-b1ceb3ec587d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.125645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.126297 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.126566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.126834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.127546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.129602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.141911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.142422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.142606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.142844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] PATH_INFO: `/v3/projects/cc753a070be64b3c807f848146433c14/users/29069b6f245b465d9a93ba3dc9e86247/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.146921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc753a070be64b3c807f848146433c14, user_id=29069b6f245b465d9a93ba3dc9e86247, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.150050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.176136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 920/1855] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:30 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:30.178262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.184853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 936/1856] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:30 2022] PUT /identity/v3/projects/cc753a070be64b3c807f848146433c14/users/29069b6f245b465d9a93ba3dc9e86247/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:30.184853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.339044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f428aac4-cc5c-4dcf-a489-127bbead7bff admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.345213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59671ad-39b0-45d7-8a8f-15e8db432105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.345932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59671ad-39b0-45d7-8a8f-15e8db432105 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.346267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59671ad-39b0-45d7-8a8f-15e8db432105 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.346566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59671ad-39b0-45d7-8a8f-15e8db432105 admin admin] PATH_INFO: `/v3/projects/5b8c594df390449bb1831b36314bf59e/users/eae3c1b69c7947108aa31019539d95a3/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.347184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d53c3-ed54-450d-88ff-2c32396d0d4b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.355623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a6002-9833-4151-874f-fab027bed968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.357482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a6002-9833-4151-874f-fab027bed968 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.357889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a6002-9833-4151-874f-fab027bed968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.359266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a6002-9833-4151-874f-fab027bed968 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.360189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a6002-9833-4151-874f-fab027bed968 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.363043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a6002-9833-4151-874f-fab027bed968 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.367089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59671ad-39b0-45d7-8a8f-15e8db432105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8c594df390449bb1831b36314bf59e, user_id=eae3c1b69c7947108aa31019539d95a3, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.371348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59671ad-39b0-45d7-8a8f-15e8db432105 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.398231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 921/1857] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:30 2022] PUT /identity/v3/projects/5b8c594df390449bb1831b36314bf59e/users/eae3c1b69c7947108aa31019539d95a3/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:30.401799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 937/1858] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:30 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:30.401935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.401935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.401935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.402195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.402590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.402731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.436390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] MFA Rules not processed for user `29069b6f245b465d9a93ba3dc9e86247`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:30.437641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:30.454324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] MFA Rules not processed for user `eae3c1b69c7947108aa31019539d95a3`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:30.458316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:30.556226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:30.609322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 922/1859] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:30.610589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.640047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 938/1860] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:30.641046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.642860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:30.678924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:30.759335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50334df5-7776-48de-8ed4-5d9e21c576a6 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.767353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed00b21-efd8-49f8-9d46-26c9a96ef537 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.768132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed00b21-efd8-49f8-9d46-26c9a96ef537 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.769033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed00b21-efd8-49f8-9d46-26c9a96ef537 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.769902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed00b21-efd8-49f8-9d46-26c9a96ef537 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.770405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b5c6a-b7f9-45c6-8bbd-3f74d55220c5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.775630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.776535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.776921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.777314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.778115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.780090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.787553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed00b21-efd8-49f8-9d46-26c9a96ef537 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.789707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed00b21-efd8-49f8-9d46-26c9a96ef537 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.802814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 923/1861] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:30.804105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.808655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 939/1862] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:30.818247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfd216-b798-4c88-888f-cd3b08efa8fc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.826350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093529e3-8d30-4bdd-b9eb-ced42c8f1feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:30.827151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093529e3-8d30-4bdd-b9eb-ced42c8f1feb admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.827433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839283cb-3909-4d37-bdd1-2490f0b33efd None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.827696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093529e3-8d30-4bdd-b9eb-ced42c8f1feb admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.828053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839283cb-3909-4d37-bdd1-2490f0b33efd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.828342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093529e3-8d30-4bdd-b9eb-ced42c8f1feb admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.829149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093529e3-8d30-4bdd-b9eb-ced42c8f1feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:30.829417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839283cb-3909-4d37-bdd1-2490f0b33efd None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.830974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 940/1863] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:46:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:30.833871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093529e3-8d30-4bdd-b9eb-ced42c8f1feb admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:30.836192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839283cb-3909-4d37-bdd1-2490f0b33efd None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:30.878411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 924/1864] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:30 2022] POST /identity/v3/projects => generated 387 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:30.880517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:30.881127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:30.881473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:30.882933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839283cb-3909-4d37-bdd1-2490f0b33efd None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:30.920718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:30.923452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:31.056768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.057753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.058174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.058605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.060948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.063668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.077765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 941/1865] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:30 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:31.078838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.091026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bf606-cb9a-4318-8859-365e5ca3925b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.100721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.101885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.102327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.102978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.104005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.104130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 925/1866] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:46:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:31.105241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.108429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.140745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 942/1867] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:31 2022] POST /identity/v3/users => generated 300 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:31.142296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.157379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea06439-17e8-469f-a069-8c94e97a8740 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.164019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.167246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.167732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.168195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] PATH_INFO: `/v3/projects/778835f2af394afd9945bbc0ac0223b0/users/28d524b038a44b2abeccc3685519e8f7/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.187704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06462769-1f08-453c-85e3-eb49bc543177 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.193289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.194324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.194959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.195480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.198444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:31.198993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] RBAC: Authorizing `identity:create_grant(project_id=778835f2af394afd9945bbc0ac0223b0, user_id=28d524b038a44b2abeccc3685519e8f7, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.204390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.266631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 943/1868] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:31 2022] PUT /identity/v3/projects/778835f2af394afd9945bbc0ac0223b0/users/28d524b038a44b2abeccc3685519e8f7/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:31.269865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.281216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:31.312151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.314385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61219a01-26fd-41f1-8173-c89c4d8433cc service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.329579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 926/1869] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:31.331415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417c4d5-80bc-41d3-8da6-5c628a3addbf None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.331852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417c4d5-80bc-41d3-8da6-5c628a3addbf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.332262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417c4d5-80bc-41d3-8da6-5c628a3addbf None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.334578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 927/1870] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:31.336631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417c4d5-80bc-41d3-8da6-5c628a3addbf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.339089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a417c4d5-80bc-41d3-8da6-5c628a3addbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:31.423398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:31.505987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a7d1-fb43-4b32-a8d5-27914b17b31f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.520608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622463e-da01-42ff-a810-67912f564b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.521270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622463e-da01-42ff-a810-67912f564b15 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.521602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622463e-da01-42ff-a810-67912f564b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.521886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622463e-da01-42ff-a810-67912f564b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.526007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622463e-da01-42ff-a810-67912f564b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.527906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622463e-da01-42ff-a810-67912f564b15 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.548591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417c4d5-80bc-41d3-8da6-5c628a3addbf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.553048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 944/1871] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:31.558269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec586f-4236-4d0e-b1f8-f929f47f09b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.559391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.559694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.561012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec586f-4236-4d0e-b1f8-f929f47f09b2 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.562489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.562929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec586f-4236-4d0e-b1f8-f929f47f09b2 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.563533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec586f-4236-4d0e-b1f8-f929f47f09b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.579005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec586f-4236-4d0e-b1f8-f929f47f09b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.581175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec586f-4236-4d0e-b1f8-f929f47f09b2 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.609861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 928/1872] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:31.620999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.623392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.624096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.636134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] MFA Rules not processed for user `28d524b038a44b2abeccc3685519e8f7`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:31.641391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:31.677989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:31.682639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:31.722309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 929/1873] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:46:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:31.722309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.749497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b59dfc8-c90b-43ce-9361-29d6e60e45dc None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.766314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.767653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.767987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.768271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] PATH_INFO: `/v3/projects/d9956f6d7d6147389698363f967ffaea/users/49c5238538ab469b9894d511c043d5da/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.794894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9956f6d7d6147389698363f967ffaea, user_id=49c5238538ab469b9894d511c043d5da, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.800011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.833636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 930/1874] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:31 2022] PUT /identity/v3/projects/d9956f6d7d6147389698363f967ffaea/users/49c5238538ab469b9894d511c043d5da/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:31.838157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.848246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 945/1875] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:31.854191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:31.870079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae54bbc-4fd1-4ea3-a5d0-61221b46cb89 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.968412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d495b-b128-4884-8ea7-5718c756ac00 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:31.973984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.974603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.974891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.975162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.975811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.977425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.979417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:31.980208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:31.980611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:31.980994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:31.982295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:31.984666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:31.990641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 931/1876] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:31.997035 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.016006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7400c-0fd2-451b-81f5-a99400ee2232 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.022771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efd11ff-26e8-4228-afd9-51260ca1b764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.023496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd11ff-26e8-4228-afd9-51260ca1b764 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.023777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd11ff-26e8-4228-afd9-51260ca1b764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.024097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efd11ff-26e8-4228-afd9-51260ca1b764 admin admin] PATH_INFO: `/v3/projects/d9956f6d7d6147389698363f967ffaea/users/49c5238538ab469b9894d511c043d5da/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.026421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd11ff-26e8-4228-afd9-51260ca1b764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9956f6d7d6147389698363f967ffaea, user_id=49c5238538ab469b9894d511c043d5da, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.030501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efd11ff-26e8-4228-afd9-51260ca1b764 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.032266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 946/1877] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:31 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:32.033252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.055212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 932/1878] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:31 2022] PUT /identity/v3/projects/d9956f6d7d6147389698363f967ffaea/users/49c5238538ab469b9894d511c043d5da/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:32.056838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.057128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.057405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.091813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:32.107839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] MFA Rules not processed for user `49c5238538ab469b9894d511c043d5da`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:32.110272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:32.115914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b4b411-f0d2-4228-b5e1-59c6c94eb34b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.131034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:32.190159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.190967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.191251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.191522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.192275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.194974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.213389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 933/1879] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:32.214936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.217096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:32.248247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 947/1880] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:32 2022] POST /identity/v3/users => generated 300 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:32.252137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.254819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:32.334006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43c5b3c-45f1-4b66-ae4d-05493575daa9 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.338294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.339153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.339669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.340175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.349789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dd7cd-08d6-430f-aff5-632169015542 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.350869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.352314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.355756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.356370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.356733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.357581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.363827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 948/1881] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:32.370535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.372104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.374384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.389829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 934/1882] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:32.392080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e751f82-6ca6-40e1-ba5a-b5f1f4a62121 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.404447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.405185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.405338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.409658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.411302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.413902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.430381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 949/1883] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:32.434268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.463622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8400b78-78c9-4536-accb-d2c1634dd5c5 service nova] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.471701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.472804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.473589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.474406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] PATH_INFO: `/v3/projects/3ec46cb7c6474674b9d8903ce30371e5/users/1ce587f3e3e244ddac3ebe2c4a754a76/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.502428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec46cb7c6474674b9d8903ce30371e5, user_id=1ce587f3e3e244ddac3ebe2c4a754a76, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.505265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.521926 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 935/1884] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:32 2022] PUT /identity/v3/projects/3ec46cb7c6474674b9d8903ce30371e5/users/1ce587f3e3e244ddac3ebe2c4a754a76/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:32.526036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.542009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42364b4e-2682-4fd8-a540-c00c5f8c8742 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.687634 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a0de6-846b-48b8-824f-463bcd3571ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.688347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a0de6-846b-48b8-824f-463bcd3571ce admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.688786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a0de6-846b-48b8-824f-463bcd3571ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.689155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a0de6-846b-48b8-824f-463bcd3571ce admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.690017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a0de6-846b-48b8-824f-463bcd3571ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.691980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a0de6-846b-48b8-824f-463bcd3571ce admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.711121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 950/1885] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:32.715227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:32.726806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89d831-97b3-4625-9dc1-d9d1c4a3f3b7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:32.732102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:32.732922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.733463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.733774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] PATH_INFO: `/v3/projects/3ec46cb7c6474674b9d8903ce30371e5/users/1ce587f3e3e244ddac3ebe2c4a754a76/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.735834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ec46cb7c6474674b9d8903ce30371e5, user_id=1ce587f3e3e244ddac3ebe2c4a754a76, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:32.738720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:32.757954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 936/1886] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:32 2022] PUT /identity/v3/projects/3ec46cb7c6474674b9d8903ce30371e5/users/1ce587f3e3e244ddac3ebe2c4a754a76/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:32.762230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:32.762519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:32.762791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:32.788403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:32.799224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] MFA Rules not processed for user `1ce587f3e3e244ddac3ebe2c4a754a76`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:32.800503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:32.886667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 951/1887] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:32.949237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.009834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd4d27f-41b9-420b-8d9d-5a39aad66306 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.014439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c3eb4-3226-4774-bce9-766c9b44b338 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.015311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3eb4-3226-4774-bce9-766c9b44b338 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.015648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3eb4-3226-4774-bce9-766c9b44b338 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.015917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c3eb4-3226-4774-bce9-766c9b44b338 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.025729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c3eb4-3226-4774-bce9-766c9b44b338 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.027146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c3eb4-3226-4774-bce9-766c9b44b338 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.031511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:33.043347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 937/1888] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:33.074855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.189284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d02c722-f3ce-462d-9101-c22b4b039bd1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.194578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.196538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.196817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.197343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] PATH_INFO: `/v3/users/28d524b038a44b2abeccc3685519e8f7` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.200000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] RBAC: Authorizing `identity:delete_user(user_id=28d524b038a44b2abeccc3685519e8f7)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.202784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.298094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28d524b038a44b2abeccc3685519e8f7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.299168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28d524b038a44b2abeccc3685519e8f7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.307997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28d524b038a44b2abeccc3685519e8f7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.308833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28d524b038a44b2abeccc3685519e8f7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.334077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28d524b038a44b2abeccc3685519e8f7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.350424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 952/1889] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:33 2022] DELETE /identity/v3/users/28d524b038a44b2abeccc3685519e8f7 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:33.355515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.356029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.356132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.373708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.375597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.377967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:33.379153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.392099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:33.454247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c58c2c-7895-4c64-9322-d677e5a7f12b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.457702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 938/1890] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:33.463312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.464044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.464384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.464743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] PATH_INFO: `/v3/users/eae3c1b69c7947108aa31019539d95a3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.466413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.466519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=eae3c1b69c7947108aa31019539d95a3)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.468460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.469205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.525506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eae3c1b69c7947108aa31019539d95a3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.531593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eae3c1b69c7947108aa31019539d95a3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.532944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732a7c1f-de1c-48c0-b41d-5aa6ba7134d7 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.536971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.537432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.537800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.538118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.540465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.542502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eae3c1b69c7947108aa31019539d95a3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.543290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eae3c1b69c7947108aa31019539d95a3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.554985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.556346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af89eab-a5e3-48c6-94f8-37951383fb9f service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.559258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eae3c1b69c7947108aa31019539d95a3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:33.566596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 939/1891] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:33.575893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 953/1892] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:33 2022] DELETE /identity/v3/users/eae3c1b69c7947108aa31019539d95a3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:33.580478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.580768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.581049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.606643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:33.609158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.682706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 940/1893] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:33.692958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:33.694487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:33.766454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355d244-13ed-40ee-b4a3-cf36f95ec1ff admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:33.770056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:33.770670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:33.770952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:33.771226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:33.780934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:33.782396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:33.793694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 954/1894] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:34.145641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.147448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.162646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.164262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.164605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5e063-e0bf-4dc5-b8bb-df3a038c7aac None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.172139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.172139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.172139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.172429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] PATH_INFO: `/v3/projects/778835f2af394afd9945bbc0ac0223b0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.173433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778835f2af394afd9945bbc0ac0223b0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:34.175396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:34.182134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4846d091-c579-4ea6-ab17-f2efefe3fcac service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.187974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.188783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.189066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.189400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] PATH_INFO: `/v3/projects/5b8c594df390449bb1831b36314bf59e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.194243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8c594df390449bb1831b36314bf59e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:34.196279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:34.242952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:34.257428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778835f2af394afd9945bbc0ac0223b0', 'request_id': 'req-8fad6f4e-c95f-432f-92d5-d337e67b6824'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.267112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778835f2af394afd9945bbc0ac0223b0', 'request_id': 'req-8fad6f4e-c95f-432f-92d5-d337e67b6824'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.267257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:34.275282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778835f2af394afd9945bbc0ac0223b0', 'request_id': 'req-8fad6f4e-c95f-432f-92d5-d337e67b6824'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.291341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8c594df390449bb1831b36314bf59e', 'request_id': 'req-cf4464a1-da79-419c-9959-a75e41e2a3dd'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.292451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] The token cache is being invalidate because project 778835f2af394afd9945bbc0ac0223b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:34.292644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.293271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.297684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 941/1895] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:34 2022] DELETE /identity/v3/projects/778835f2af394afd9945bbc0ac0223b0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:34.297989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8c594df390449bb1831b36314bf59e', 'request_id': 'req-cf4464a1-da79-419c-9959-a75e41e2a3dd'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.299082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.300774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.306638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8c594df390449bb1831b36314bf59e', 'request_id': 'req-cf4464a1-da79-419c-9959-a75e41e2a3dd'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.318668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] The token cache is being invalidate because project 5b8c594df390449bb1831b36314bf59e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:34.318962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.319618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.321124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 955/1896] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:34 2022] DELETE /identity/v3/projects/5b8c594df390449bb1831b36314bf59e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:34.325059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:34.327218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:34.374269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad6f4e-c95f-432f-92d5-d337e67b6824 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.406890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4464a1-da79-419c-9959-a75e41e2a3dd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:34.418591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.419055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.419945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.420229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] PATH_INFO: `/v3/users/49c5238538ab469b9894d511c043d5da` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.433222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c5238538ab469b9894d511c043d5da)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:34.435268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:34.453490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:34.453875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:34.453875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:34.454135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] PATH_INFO: `/v3/users/1ce587f3e3e244ddac3ebe2c4a754a76` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:34.455318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce587f3e3e244ddac3ebe2c4a754a76)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:34.457278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:34.520673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c5238538ab469b9894d511c043d5da'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.522209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c5238538ab469b9894d511c043d5da'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.525956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce587f3e3e244ddac3ebe2c4a754a76'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.526705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce587f3e3e244ddac3ebe2c4a754a76'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.536177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c5238538ab469b9894d511c043d5da'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.536555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce587f3e3e244ddac3ebe2c4a754a76'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.537522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce587f3e3e244ddac3ebe2c4a754a76'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.537801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c5238538ab469b9894d511c043d5da'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.548848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce587f3e3e244ddac3ebe2c4a754a76'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.556541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c5238538ab469b9894d511c043d5da'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:34.563756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:34.578760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 956/1897] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:34 2022] DELETE /identity/v3/users/49c5238538ab469b9894d511c043d5da => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:34.585324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 942/1898] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:34 2022] DELETE /identity/v3/users/1ce587f3e3e244ddac3ebe2c4a754a76 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:35.219168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.220872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.226914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.228913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.301672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:35.336691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c140a-a192-4b93-90df-b264ffcb10dd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.373543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc7bc3-c784-4882-9469-63d95bfe3630 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.378569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.379302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.379485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.379765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] PATH_INFO: `/v3/projects/d9956f6d7d6147389698363f967ffaea` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.381160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9956f6d7d6147389698363f967ffaea)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.383730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.430105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.430922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.431303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.431675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] PATH_INFO: `/v3/projects/3ec46cb7c6474674b9d8903ce30371e5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.432991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ec46cb7c6474674b9d8903ce30371e5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.435256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.449893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9956f6d7d6147389698363f967ffaea', 'request_id': 'req-9479ea40-0b07-49fd-8f71-da2a9783b332'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.461998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9956f6d7d6147389698363f967ffaea', 'request_id': 'req-9479ea40-0b07-49fd-8f71-da2a9783b332'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.472737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9956f6d7d6147389698363f967ffaea', 'request_id': 'req-9479ea40-0b07-49fd-8f71-da2a9783b332'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.495197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] The token cache is being invalidate because project d9956f6d7d6147389698363f967ffaea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:35.495873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.496767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9479ea40-0b07-49fd-8f71-da2a9783b332 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.499218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 943/1899] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:35 2022] DELETE /identity/v3/projects/d9956f6d7d6147389698363f967ffaea => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:35.505077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082a4b7-a795-4595-baa4-c1aaabffb797 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.505367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082a4b7-a795-4595-baa4-c1aaabffb797 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.505683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082a4b7-a795-4595-baa4-c1aaabffb797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.512774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ec46cb7c6474674b9d8903ce30371e5', 'request_id': 'req-b73396d1-2a43-487c-972f-916301afe75a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.523856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ec46cb7c6474674b9d8903ce30371e5', 'request_id': 'req-b73396d1-2a43-487c-972f-916301afe75a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.531845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-8082a4b7-a795-4595-baa4-c1aaabffb797 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:35.533912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8082a4b7-a795-4595-baa4-c1aaabffb797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.534676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ec46cb7c6474674b9d8903ce30371e5', 'request_id': 'req-b73396d1-2a43-487c-972f-916301afe75a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.552578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] The token cache is being invalidate because project 3ec46cb7c6474674b9d8903ce30371e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:35.552812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.553494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b73396d1-2a43-487c-972f-916301afe75a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:35.556786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 957/1900] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:35 2022] DELETE /identity/v3/projects/3ec46cb7c6474674b9d8903ce30371e5 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:35.561549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217eb86-f2a6-4ae0-a1ba-66267e82db74 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.561868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217eb86-f2a6-4ae0-a1ba-66267e82db74 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.562274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e217eb86-f2a6-4ae0-a1ba-66267e82db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.587998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e217eb86-f2a6-4ae0-a1ba-66267e82db74 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:35.589403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e217eb86-f2a6-4ae0-a1ba-66267e82db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.621311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 944/1901] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:35.630058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.630394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.630675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.654632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:35.655684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.665602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 945/1902] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:35.673601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 958/1903] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:35.673736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.677647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.677967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.678400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.680412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff60f7f-e5a8-426b-98d8-6c6412b2558d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.685584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.686181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.686661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.686943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.687598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:35.687905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.689509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.698203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:35.699566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:35.705631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 946/1904] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:35.707649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.708939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 959/1905] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:35.713033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.715906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:35.722219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3647d88-a2be-4eeb-b97a-84373bce40fa admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.725280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c153c4e-2e4a-44cb-94a8-b51a46030ec4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.729137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.730032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.730551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.731220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.731801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.732091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.732491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.732775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.733107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.735340 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:35.735340 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.735550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.736057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.739888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:35.753006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 960/1906] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:35.757363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.767367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 947/1907] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:35.769250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2445d8e-4f1f-4d5a-9b21-52cf8323b940 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.772570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.855603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892b974f-30aa-4c61-a6c4-5b12740c1c10 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:35.860591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.861289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.861666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.861847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:35.861958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.862521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:35.862627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.862835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:35.863076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:35.864184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:35.865925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.866223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:35.905121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 948/1908] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:35.910586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.917821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 961/1909] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/projects => generated 393 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:35.922207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:35.922688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a02b769-e180-4a06-a472-09f74ae051f0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.009927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.011585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.012197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.012664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.014520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.016891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.017368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20811d4-2aac-4036-b6c9-45f3d330e0bd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.023171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.023791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.024156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.024341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.024983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.027048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.034460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 949/1910] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.038880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.055015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f57d32-9ee7-4f96-8a01-6dce408eedd3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.063762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.064647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.065078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.065504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.066519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.068266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 962/1911] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:35 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:36.069401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.073820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.085245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 950/1912] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.089819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.093196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9414c-ee42-4752-9b5a-bea64b1fe518 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.098737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.100123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.100487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.100814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.101485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.103348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.111533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58cf50-659a-4007-9c34-3c7a22778041 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.117098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 963/1913] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:36.120769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4fdd4-9d6f-463f-bf25-7183441bb90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.120923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4fdd4-9d6f-463f-bf25-7183441bb90e admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.121153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4fdd4-9d6f-463f-bf25-7183441bb90e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.121437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4fdd4-9d6f-463f-bf25-7183441bb90e admin admin] PATH_INFO: `/v3/projects/e872ac76e11e43ed9adabf111107b3c4/users/4951d9e81f7347efabff3c7abd1c3007/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.123377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.143062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07ee25-503d-4787-a694-2591dd948264 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.148507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4fdd4-9d6f-463f-bf25-7183441bb90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e872ac76e11e43ed9adabf111107b3c4, user_id=4951d9e81f7347efabff3c7abd1c3007, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.151712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4fdd4-9d6f-463f-bf25-7183441bb90e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.156013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.156631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.156913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.157182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.157878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.159472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.171512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 964/1915] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.173773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 951/1915] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/e872ac76e11e43ed9adabf111107b3c4/users/4951d9e81f7347efabff3c7abd1c3007/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:36.175450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.180204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.180492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.180771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.191987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7c564-fe71-4230-bf44-ea881e841236 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.212066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] MFA Rules not processed for user `4951d9e81f7347efabff3c7abd1c3007`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:36.213328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:36.233027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:36.248372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:36.273874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc3c273-5cd1-47cc-9609-2ccd5f1af5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.289154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c273-5cd1-47cc-9609-2ccd5f1af5c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.289154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c273-5cd1-47cc-9609-2ccd5f1af5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.289154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc3c273-5cd1-47cc-9609-2ccd5f1af5c7 admin admin] PATH_INFO: `/v3/projects/a15ee79400484914b04799525823411e/users/9eefeef6fb974dbe8ca4bebe349ca349/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.292292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3c273-5cd1-47cc-9609-2ccd5f1af5c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a15ee79400484914b04799525823411e, user_id=9eefeef6fb974dbe8ca4bebe349ca349, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.295077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc3c273-5cd1-47cc-9609-2ccd5f1af5c7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.304184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 952/1916] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:36.308100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.308923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 965/1917] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/a15ee79400484914b04799525823411e/users/9eefeef6fb974dbe8ca4bebe349ca349/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:36.319090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff88f08-b822-4636-a060-c46299717a19 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.319275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916575c-034c-49fb-b77a-015a4dd5b1b3 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.319383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff88f08-b822-4636-a060-c46299717a19 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.319636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff88f08-b822-4636-a060-c46299717a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.349474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-dff88f08-b822-4636-a060-c46299717a19 None None] MFA Rules not processed for user `9eefeef6fb974dbe8ca4bebe349ca349`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:36.350719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-dff88f08-b822-4636-a060-c46299717a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:36.379999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.380615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.380958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.381251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.381913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.383845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.415019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 953/1918] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:36.416412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.420557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 966/1919] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:36.425019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff88f08-b822-4636-a060-c46299717a19 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.436649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff88f08-b822-4636-a060-c46299717a19 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.486031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3304d2ae-9cdd-4161-9f1a-b49decf77eb4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.490914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.491529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.491902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.492179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.492820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.494798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.506406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.507031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.507313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.507587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.508254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.510334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.521208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 954/1920] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:36.525431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.541957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50353bcd-a763-415c-ad63-d03da6068434 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.549366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.550195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 967/1921] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:36.550318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.550318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.550632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.551285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.553180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.553698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.566612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 955/1922] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.570735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.581337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909a569-4ea3-45b5-973e-e5aa7acbc396 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.605639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:36.631799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:36.650272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df2fc9-d444-4ff7-847b-a755e46e47f7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.655680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.656306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.656586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.656862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.657513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.659649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.661164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.662049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.662322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.662663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] PATH_INFO: `/v3/projects/b94d3a39c6ba4da486a0582edaeeaa0a/users/b787218bb73749beb47702f56a062b35/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.682068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b94d3a39c6ba4da486a0582edaeeaa0a, user_id=b787218bb73749beb47702f56a062b35, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.685791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 968/1923] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/users => generated 303 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:36.688702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.689483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.705964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264aab0-4c4c-4aaa-b8fe-55c0d6ae3eac admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.709135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 956/1924] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/b94d3a39c6ba4da486a0582edaeeaa0a/users/b787218bb73749beb47702f56a062b35/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:36.710084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.774259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44faaf7d-fd9f-484a-8895-b55f732e56d5 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.778922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.779428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.779555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.779792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.780064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.780160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.780352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.780646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.780787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.781292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.782340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.782976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.792461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 957/1925] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.793374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 969/1926] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.796728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.797829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.807420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58da023b-3702-4341-a4be-8aa3f79bb622 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.807565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a59e6e-5774-40f1-baa6-d79c5cf01802 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.813229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.813482 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6fafac-7aec-412c-bcb8-f40ea56f79ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.813936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.814170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fafac-7aec-412c-bcb8-f40ea56f79ab admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.814284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.814450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fafac-7aec-412c-bcb8-f40ea56f79ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.814546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] PATH_INFO: `/v3/projects/9dba3ab8d57344c48e7df7f3a1ec339d/users/1a23597d83ba4138b824ec4ef671bd3e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.814726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6fafac-7aec-412c-bcb8-f40ea56f79ab admin admin] PATH_INFO: `/v3/projects/b94d3a39c6ba4da486a0582edaeeaa0a/users/b787218bb73749beb47702f56a062b35/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.816647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fafac-7aec-412c-bcb8-f40ea56f79ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b94d3a39c6ba4da486a0582edaeeaa0a, user_id=b787218bb73749beb47702f56a062b35, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.819340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6fafac-7aec-412c-bcb8-f40ea56f79ab admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.830517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dba3ab8d57344c48e7df7f3a1ec339d, user_id=1a23597d83ba4138b824ec4ef671bd3e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.833222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.834891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 958/1927] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/b94d3a39c6ba4da486a0582edaeeaa0a/users/b787218bb73749beb47702f56a062b35/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:36.846307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.846594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.846872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.847881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 970/1928] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/9dba3ab8d57344c48e7df7f3a1ec339d/users/1a23597d83ba4138b824ec4ef671bd3e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:36.852136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.875439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] MFA Rules not processed for user `b787218bb73749beb47702f56a062b35`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:36.876595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:36.915062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6367b7-4fe2-4f24-a62a-2a263dd01401 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.920420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.921034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.921315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.921616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.922263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.923846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.934001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 971/1929] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:36.938849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.948904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c714b46-efe7-4310-823f-73ca9377d8b4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:36.952668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 959/1930] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:36.954119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eda3b8-fc96-4f38-8603-708eef0e9ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:36.954873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda3b8-fc96-4f38-8603-708eef0e9ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.955179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda3b8-fc96-4f38-8603-708eef0e9ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.955453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eda3b8-fc96-4f38-8603-708eef0e9ad7 admin admin] PATH_INFO: `/v3/projects/9dba3ab8d57344c48e7df7f3a1ec339d/users/1a23597d83ba4138b824ec4ef671bd3e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.957389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda3b8-fc96-4f38-8603-708eef0e9ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dba3ab8d57344c48e7df7f3a1ec339d, user_id=1a23597d83ba4138b824ec4ef671bd3e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:36.960089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eda3b8-fc96-4f38-8603-708eef0e9ad7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:36.965452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:36.967492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:36.980428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 972/1931] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:36 2022] PUT /identity/v3/projects/9dba3ab8d57344c48e7df7f3a1ec339d/users/1a23597d83ba4138b824ec4ef671bd3e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:36.983417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:36.983711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:36.983978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:36.988244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:37.022214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] MFA Rules not processed for user `1a23597d83ba4138b824ec4ef671bd3e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:37.023591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:37.039455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ebc12-ac8f-4ff1-bb45-36a2251ea549 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:37.044773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:37.095823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:37.096453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:37.096735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:37.097007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:37.111610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 973/1932] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:36 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:37.125195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:37.127264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:37.139180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d0588c-3d1a-4735-b89f-64ddf7993a97 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:37.142827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:37.143435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:37.143719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:37.143995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:37.153211 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:37.154701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:37.165089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 974/1933] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:37 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:37.165339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:37.167004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:37.180077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 960/1934] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:36 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:37.223459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:37.237466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2086b750-d7cf-42f3-b853-d72234ad220c service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:37.242796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61549d06-1a9f-4bd9-a54f-09703c7a0f97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:37.242962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61549d06-1a9f-4bd9-a54f-09703c7a0f97 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:37.243249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61549d06-1a9f-4bd9-a54f-09703c7a0f97 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:37.243530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61549d06-1a9f-4bd9-a54f-09703c7a0f97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:37.337026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61549d06-1a9f-4bd9-a54f-09703c7a0f97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:37.340301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61549d06-1a9f-4bd9-a54f-09703c7a0f97 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:37.354003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 975/1935] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:37 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:37.594289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:37.686270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9d19-68c9-47b0-bba9-fc195cb20bf3 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:37.691045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:37.691918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:37.692149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:37.692435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] PATH_INFO: `/v3/users/4951d9e81f7347efabff3c7abd1c3007` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:37.693720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4951d9e81f7347efabff3c7abd1c3007)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:37.695995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:37.725520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:37.762423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4951d9e81f7347efabff3c7abd1c3007'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:37.764046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4951d9e81f7347efabff3c7abd1c3007'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:37.773503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4951d9e81f7347efabff3c7abd1c3007'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:37.774541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4951d9e81f7347efabff3c7abd1c3007'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:37.785695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4951d9e81f7347efabff3c7abd1c3007'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:37.808451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 961/1936] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:37 2022] DELETE /identity/v3/users/4951d9e81f7347efabff3c7abd1c3007 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:37.814048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:37.814426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:37.814731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:37.834212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:37.835420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:37.894426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 976/1937] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:37.904241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:37.905771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:37.963311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd5a339-259c-4db3-a491-600dee972879 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:37.966936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:37.967545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:37.967826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:37.968103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:37.977026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:37.978484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:37.988670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 962/1938] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:38.270443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:38.271896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:38.302122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d34d-40e3-455a-a4d4-607572cfc092 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:38.307451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:38.308089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:38.308370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:38.308643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] PATH_INFO: `/v3/projects/e872ac76e11e43ed9adabf111107b3c4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:38.310350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e872ac76e11e43ed9adabf111107b3c4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:38.312544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:38.352208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:38.362880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e872ac76e11e43ed9adabf111107b3c4', 'request_id': 'req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.369813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e872ac76e11e43ed9adabf111107b3c4', 'request_id': 'req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.379332 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e872ac76e11e43ed9adabf111107b3c4', 'request_id': 'req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.390972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] The token cache is being invalidate because project e872ac76e11e43ed9adabf111107b3c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:38.391261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.391942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.393432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 977/1939] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:38 2022] DELETE /identity/v3/projects/e872ac76e11e43ed9adabf111107b3c4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:38.397480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:38.407515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01c90d-acc3-4ee0-8293-3a102feb356f service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:38.466933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:38.467575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:38.467849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:38.468117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] PATH_INFO: `/v3/users/b787218bb73749beb47702f56a062b35` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:38.469268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b787218bb73749beb47702f56a062b35)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:38.471282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:38.544684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:38.546843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b787218bb73749beb47702f56a062b35'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.548478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b787218bb73749beb47702f56a062b35'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.559236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b787218bb73749beb47702f56a062b35'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.560554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b787218bb73749beb47702f56a062b35'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.574404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49cdf808-7ae7-499f-be57-31c3ee76d106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b787218bb73749beb47702f56a062b35'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:38.603842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 963/1940] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:38 2022] DELETE /identity/v3/users/b787218bb73749beb47702f56a062b35 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:39.107826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.109547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:39.123306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb5ecd6-3ebf-47ed-a3fe-4ea979a14d57 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.187037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.187577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.187859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.188165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] PATH_INFO: `/v3/projects/b94d3a39c6ba4da486a0582edaeeaa0a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.189293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b94d3a39c6ba4da486a0582edaeeaa0a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.191264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.240066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b94d3a39c6ba4da486a0582edaeeaa0a', 'request_id': 'req-887b28f1-1cb6-4274-8ac2-d3899351aa87'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:39.245936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b94d3a39c6ba4da486a0582edaeeaa0a', 'request_id': 'req-887b28f1-1cb6-4274-8ac2-d3899351aa87'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:39.255872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b94d3a39c6ba4da486a0582edaeeaa0a', 'request_id': 'req-887b28f1-1cb6-4274-8ac2-d3899351aa87'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:39.272709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] The token cache is being invalidate because project b94d3a39c6ba4da486a0582edaeeaa0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:39.273089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:39.273740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-887b28f1-1cb6-4274-8ac2-d3899351aa87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:39.275276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 978/1941] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:39 2022] DELETE /identity/v3/projects/b94d3a39c6ba4da486a0582edaeeaa0a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:39.280100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.280465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.280741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.302290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:39.303520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:39.373651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 964/1942] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:39.381604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.382208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.382491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.404649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:39.405986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:39.418898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.426101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 979/1943] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:39.430666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9b56-cad6-4589-b39d-00c3e235a189 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.438016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.438578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.438858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.439132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.442546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:39.443023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.445659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.465058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.475545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a403ac-5486-430c-9406-3c689f9f4eac None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.476606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 965/1944] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:39.482704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.482704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.482704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.483036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.483761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.486003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.524553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.532820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 980/1945] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:39 2022] POST /identity/v3/projects => generated 361 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:39.537963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9526c8f4-20dc-4874-af37-039425b8e61c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.620472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.622025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.623018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.623518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.624393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.626640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.666653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 966/1946] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:39 2022] POST /identity/v3/users => generated 281 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:39.671176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.684498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ded535-128f-4d48-bbcd-cfb4ab5703b7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.690314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.690952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.691628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.691982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.692823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.694564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.706453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 981/1947] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:39.710946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.726598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df397a8-5dc0-4021-8c95-e4fb403bfb5a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.732903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23492be6-09d2-4473-8a35-a74417cbb32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.734067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23492be6-09d2-4473-8a35-a74417cbb32c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.734353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23492be6-09d2-4473-8a35-a74417cbb32c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.734632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23492be6-09d2-4473-8a35-a74417cbb32c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.735594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23492be6-09d2-4473-8a35-a74417cbb32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.736953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23492be6-09d2-4473-8a35-a74417cbb32c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.753243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 967/1948] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:39.758613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:39.770302 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5536ce-8757-493b-9a6a-271c4aba0280 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:39.777290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:39.777993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.778414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.779839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] PATH_INFO: `/v3/projects/e703e982646149ec810f22770a3de0ec/users/b1e50256b72845e0925e45196d58fcff/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.812148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:39.815877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e703e982646149ec810f22770a3de0ec, user_id=b1e50256b72845e0925e45196d58fcff, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:39.829923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:39.858062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 982/1949] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:39 2022] PUT /identity/v3/projects/e703e982646149ec810f22770a3de0ec/users/b1e50256b72845e0925e45196d58fcff/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:39.869451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:39.869790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:39.870064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:39.905777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] MFA Rules not processed for user `b1e50256b72845e0925e45196d58fcff`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:39.907106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:39.928027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:40.019150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 968/1950] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:40.153262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:40.155438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:40.248588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:40.250054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:40.284494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95068ea3-d5ff-424b-a353-6e48383ece14 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:40.289938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:40.291102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.292145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.292502 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] PATH_INFO: `/v3/users/9a71ceb0c6a54543beaa7ba7a94375d3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.294848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a71ceb0c6a54543beaa7ba7a94375d3)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:40.296868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:40.333912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a3976e-ed5f-47b9-b600-c24fc4cc388e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:40.338227 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:40.338845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.339130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.339406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.349863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:40.351306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:40.353052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a71ceb0c6a54543beaa7ba7a94375d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.353890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a71ceb0c6a54543beaa7ba7a94375d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.362713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 969/1951] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:40.363736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a71ceb0c6a54543beaa7ba7a94375d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.365956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a71ceb0c6a54543beaa7ba7a94375d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.379022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:40.391322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f36f448f-6169-4cac-98a0-005568729f15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a71ceb0c6a54543beaa7ba7a94375d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.401613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c73ddfb-5119-4680-a396-18437df1ed75 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:40.407732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:40.408366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.408651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.408929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] PATH_INFO: `/v3/users/b1e50256b72845e0925e45196d58fcff` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.410129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e50256b72845e0925e45196d58fcff)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:40.412452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:40.418035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 983/1952] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:40 2022] DELETE /identity/v3/users/9a71ceb0c6a54543beaa7ba7a94375d3 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:40.422730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.423249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.423600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.457520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:40.459575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:40.482664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e50256b72845e0925e45196d58fcff'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.484056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e50256b72845e0925e45196d58fcff'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.492991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e50256b72845e0925e45196d58fcff'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.494068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e50256b72845e0925e45196d58fcff'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.511115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942545f7-4725-408f-bda4-b0b7383617ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e50256b72845e0925e45196d58fcff'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:40.573425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 970/1953] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:40 2022] DELETE /identity/v3/users/b1e50256b72845e0925e45196d58fcff => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:40.574984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.575381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.575751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.577299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 984/1954] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:40.601237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:40.601436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:40.603120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:40.603428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:40.626637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:40.673418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:40.745824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3bbae-3280-4b16-a3be-7f963cb6965d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:40.752378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 971/1955] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:40.764502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:40.884095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a5296c-ec45-4df9-bd89-2e9fc7073227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:40.884744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5296c-ec45-4df9-bd89-2e9fc7073227 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.885029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5296c-ec45-4df9-bd89-2e9fc7073227 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.890439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a5296c-ec45-4df9-bd89-2e9fc7073227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.912615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5296c-ec45-4df9-bd89-2e9fc7073227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:40.916615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a5296c-ec45-4df9-bd89-2e9fc7073227 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:40.933064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2824c55-b910-4250-a0a2-575dc9748a94 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:40.939701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98af3ce-db01-4c99-8b2b-cd96693bff47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:40.940408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af3ce-db01-4c99-8b2b-cd96693bff47 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:40.942161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af3ce-db01-4c99-8b2b-cd96693bff47 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:40.942161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98af3ce-db01-4c99-8b2b-cd96693bff47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:40.948343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 985/1956] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:40.978957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98af3ce-db01-4c99-8b2b-cd96693bff47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:40.984320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98af3ce-db01-4c99-8b2b-cd96693bff47 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:41.013404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 972/1957] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:41.028958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def061f-7e7c-4a4e-994e-bbccc3bfada2 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.029987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def061f-7e7c-4a4e-994e-bbccc3bfada2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.030552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0def061f-7e7c-4a4e-994e-bbccc3bfada2 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.032961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 986/1958] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:46:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:46:41.048653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.048941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.049223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.096411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:41.099688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:41.242478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 973/1959] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:46:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:41.334783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def061f-7e7c-4a4e-994e-bbccc3bfada2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:41.336551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0def061f-7e7c-4a4e-994e-bbccc3bfada2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:41.360721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0def061f-7e7c-4a4e-994e-bbccc3bfada2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:41.367109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:41.367694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.367971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.368675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.381057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:41.383238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:41.396400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 987/1960] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:41.756032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:41.757624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:41.782826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49f2b6-b281-4265-b024-99c43ccac724 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:41.796128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:41.796780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.797065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.797341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] PATH_INFO: `/v3/projects/e703e982646149ec810f22770a3de0ec` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.798605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e703e982646149ec810f22770a3de0ec)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:41.800597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:41.838891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:41.840358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:41.857388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e703e982646149ec810f22770a3de0ec', 'request_id': 'req-c2452e96-5a5f-4722-b2fb-38b52486b579'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:41.865009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e703e982646149ec810f22770a3de0ec', 'request_id': 'req-c2452e96-5a5f-4722-b2fb-38b52486b579'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:41.866613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c480b520-a97f-4c25-b82b-630dcd64a5b3 service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:41.873445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e703e982646149ec810f22770a3de0ec', 'request_id': 'req-c2452e96-5a5f-4722-b2fb-38b52486b579'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:41.875375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:41.889153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] The token cache is being invalidate because project e703e982646149ec810f22770a3de0ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:41.889442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:41.890140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c2452e96-5a5f-4722-b2fb-38b52486b579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:41.891700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 974/1961] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:41 2022] DELETE /identity/v3/projects/e703e982646149ec810f22770a3de0ec => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:41.896366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8222-6025-46d7-b367-1880032898df None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.896654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8222-6025-46d7-b367-1880032898df None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.896947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023d8222-6025-46d7-b367-1880032898df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.918120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-023d8222-6025-46d7-b367-1880032898df None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:41.919390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-023d8222-6025-46d7-b367-1880032898df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:41.928159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:41.928893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.929178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.929448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] PATH_INFO: `/v3/projects/c6b1689d55c24d10b48e42a56fafba12` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.930587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b1689d55c24d10b48e42a56fafba12)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:41.932475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:41.982430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 975/1962] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:41.989863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:41.990160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:41.990441 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:41.992756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b1689d55c24d10b48e42a56fafba12', 'request_id': 'req-e6ada0a3-8265-428e-95d0-afc02be950df'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:41.999210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b1689d55c24d10b48e42a56fafba12', 'request_id': 'req-e6ada0a3-8265-428e-95d0-afc02be950df'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.001935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:42.010585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b1689d55c24d10b48e42a56fafba12', 'request_id': 'req-e6ada0a3-8265-428e-95d0-afc02be950df'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.017468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:42.018897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.026890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] The token cache is being invalidate because project c6b1689d55c24d10b48e42a56fafba12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:42.027345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.028093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.030635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 988/1963] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:41 2022] DELETE /identity/v3/projects/c6b1689d55c24d10b48e42a56fafba12 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:42.034290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.035774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.089068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 976/1964] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:42.094053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.103336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d315d9-1dba-4a69-9c75-71672292b90c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.109722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.110333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.110617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.110890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.111541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:42.111844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.113471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.117512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ada0a3-8265-428e-95d0-afc02be950df admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.122929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.123379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.123798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.124051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] PATH_INFO: `/v3/users/ec4376b2d18f4a8c80f689b0468d67a6` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.125143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec4376b2d18f4a8c80f689b0468d67a6)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.127255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.129631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 977/1965] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:42.130431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.140468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132c36c-4d46-4f0a-80b5-534422414247 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.145264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d335527-49df-4412-93a5-344d320758de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.145895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335527-49df-4412-93a5-344d320758de admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.146179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335527-49df-4412-93a5-344d320758de admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.146452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d335527-49df-4412-93a5-344d320758de admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.147072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d335527-49df-4412-93a5-344d320758de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.149006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d335527-49df-4412-93a5-344d320758de admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.187324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 978/1966] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:42 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:42.188616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d335527-49df-4412-93a5-344d320758de admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.189835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec4376b2d18f4a8c80f689b0468d67a6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.190969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec4376b2d18f4a8c80f689b0468d67a6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.199943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec4376b2d18f4a8c80f689b0468d67a6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.201142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec4376b2d18f4a8c80f689b0468d67a6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.220349 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec4376b2d18f4a8c80f689b0468d67a6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.255186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 989/1967] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:42 2022] DELETE /identity/v3/users/ec4376b2d18f4a8c80f689b0468d67a6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:42.293180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:42.315486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d335527-49df-4412-93a5-344d320758de admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.414498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.415074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.415352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.416117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.416928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.419607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.451834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 979/1968] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:42 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:42.452968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.455103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.477000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93b3fa-8aba-48e2-a8aa-6d3ed2787410 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.484088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.484873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.485370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.485936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.486491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.489218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.519418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.520986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 990/1969] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:42.536512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c45d31-7c4a-4fbe-be0c-4a4092a35b64 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.547636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.549195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.554311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.554840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.557113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.558100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.560178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.565935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.579134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc411ff0-2b59-4d67-bc97-c8b7a66e756e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.591072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.592145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.592445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.592712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] PATH_INFO: `/v3/users/9eefeef6fb974dbe8ca4bebe349ca349` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.593070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 980/1970] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:42 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:42.597289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.597709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eefeef6fb974dbe8ca4bebe349ca349)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.600020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.617750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929ee54f-c0bc-4fe8-b2a6-d64ece4e78e4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:42.627390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadaa93d-a19a-4316-8bce-9cdc90aab3ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:42.628710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa93d-a19a-4316-8bce-9cdc90aab3ac admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.629192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa93d-a19a-4316-8bce-9cdc90aab3ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.629732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadaa93d-a19a-4316-8bce-9cdc90aab3ac admin admin] PATH_INFO: `/v3/projects/9d506ccfc1464224912d2b550f40ef09/users/466563f90b0d46d69dbe2038c11d5679/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.679704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa93d-a19a-4316-8bce-9cdc90aab3ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d506ccfc1464224912d2b550f40ef09, user_id=466563f90b0d46d69dbe2038c11d5679, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:42.686623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadaa93d-a19a-4316-8bce-9cdc90aab3ac admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:42.716130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 981/1971] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:42 2022] PUT /identity/v3/projects/9d506ccfc1464224912d2b550f40ef09/users/466563f90b0d46d69dbe2038c11d5679/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:42.726890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.727663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.727663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.732508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eefeef6fb974dbe8ca4bebe349ca349'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.733780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eefeef6fb974dbe8ca4bebe349ca349'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.737892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:INSERT=3,keystone:DELETE=18,keystone:SELECT=79 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:42.752464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eefeef6fb974dbe8ca4bebe349ca349'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.753137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eefeef6fb974dbe8ca4bebe349ca349'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.777457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-999bca7e-5443-421d-8af1-daebac3578c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eefeef6fb974dbe8ca4bebe349ca349'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:42.797423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] MFA Rules not processed for user `466563f90b0d46d69dbe2038c11d5679`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:42.798722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.811316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 991/1972] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:42 2022] DELETE /identity/v3/users/9eefeef6fb974dbe8ca4bebe349ca349 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:42.818122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:42.818436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:42.818718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:42.866836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:42.872814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.959505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 992/1973] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:42.971696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 982/1974] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:42.976017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:42.977972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:42.998425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca613ec-d4e6-432b-a199-4c749fdb9d06 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.005335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.006573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.007439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.008080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.008720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.008983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.010142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.012566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.052966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 993/1975] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:42 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:43.059390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.145537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:43.155364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d21960-9f65-4b5a-9f0a-1e7c9945edcc None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.218341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b61de0-861c-40ca-bf54-4a8422e7ddfa admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.228154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.228154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.228154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.228154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.228816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.230929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.249956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.250796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.251094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.251431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.274673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 994/1976] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:43.280941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.285622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.287094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.299264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 983/1977] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:43.304866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d8a7c-e86d-4cd7-8c52-9d8145ee11af admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.311493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.312131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.312460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.312673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.314016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.316038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.331733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 995/1978] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:43.335401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.345626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1701f98b-7415-4980-b240-1f6a898f13e9 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.353132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.372786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.376681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.377391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.378463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.382406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.402244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 984/1979] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:43 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:43.403415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.420080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49947d0e-5acf-4405-84dc-fbf535b66d3c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.429877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533cd88-0787-4034-8ab3-2a9bfeee6496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.431059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533cd88-0787-4034-8ab3-2a9bfeee6496 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.431524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533cd88-0787-4034-8ab3-2a9bfeee6496 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.433150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533cd88-0787-4034-8ab3-2a9bfeee6496 admin admin] PATH_INFO: `/v3/projects/f26172d8328f4be493537cb3ce2ad2d5/users/bb862a36f1cf4a9880c60134d77c934f/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.481218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533cd88-0787-4034-8ab3-2a9bfeee6496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26172d8328f4be493537cb3ce2ad2d5, user_id=bb862a36f1cf4a9880c60134d77c934f, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.486961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533cd88-0787-4034-8ab3-2a9bfeee6496 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.501475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:43.506327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.508566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.519709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 996/1980] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:43 2022] PUT /identity/v3/projects/f26172d8328f4be493537cb3ce2ad2d5/users/bb862a36f1cf4a9880c60134d77c934f/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:43.521387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.521862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.522146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.539286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29121832-269c-48da-b60a-d46f39a65b81 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.572867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] MFA Rules not processed for user `bb862a36f1cf4a9880c60134d77c934f`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:43.574188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.642988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.643634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.643919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.644224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] PATH_INFO: `/v3/projects/75aad4108f0d45d6a8da5e35103ef348` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.645323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75aad4108f0d45d6a8da5e35103ef348)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.647309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.669961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 997/1981] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:43.671719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.672834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.690034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d1f2a3-a1c3-418c-b3c7-f9ddebcaec0e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.695044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:43.695701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.696028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.696307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] PATH_INFO: `/v3/users/f03b6ec092d041ada29b0b92cc70dc5a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.697477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03b6ec092d041ada29b0b92cc70dc5a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:43.699475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:43.707215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75aad4108f0d45d6a8da5e35103ef348', 'request_id': 'req-506a4c4d-879c-422e-b0f7-c6994a363d79'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.714065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75aad4108f0d45d6a8da5e35103ef348', 'request_id': 'req-506a4c4d-879c-422e-b0f7-c6994a363d79'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.721712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75aad4108f0d45d6a8da5e35103ef348', 'request_id': 'req-506a4c4d-879c-422e-b0f7-c6994a363d79'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.743061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] The token cache is being invalidate because project 75aad4108f0d45d6a8da5e35103ef348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:43.743283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.744200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.749650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 985/1982] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:43 2022] DELETE /identity/v3/projects/75aad4108f0d45d6a8da5e35103ef348 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:43.751533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:43.752985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.757976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03b6ec092d041ada29b0b92cc70dc5a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.758929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03b6ec092d041ada29b0b92cc70dc5a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.769745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03b6ec092d041ada29b0b92cc70dc5a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.770391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03b6ec092d041ada29b0b92cc70dc5a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.795016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:43.801694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-72df2bf7-9377-465d-9fdc-8ba5b44d59f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03b6ec092d041ada29b0b92cc70dc5a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:43.821392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 998/1983] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:43 2022] DELETE /identity/v3/users/f03b6ec092d041ada29b0b92cc70dc5a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:43.822961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2537-ae68-4c08-8891-23f9b40bc753 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.823402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2537-ae68-4c08-8891-23f9b40bc753 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.824431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ce2537-ae68-4c08-8891-23f9b40bc753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.846755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-36ce2537-ae68-4c08-8891-23f9b40bc753 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:43.847998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-36ce2537-ae68-4c08-8891-23f9b40bc753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:43.894569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506a4c4d-879c-422e-b0f7-c6994a363d79 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:43.959783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 999/1984] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:43.965636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fb693-6bb7-4248-9157-e54ef40de94c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:43.967695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fb693-6bb7-4248-9157-e54ef40de94c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:43.968085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fb693-6bb7-4248-9157-e54ef40de94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:43.983772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:44.001400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e91fb693-6bb7-4248-9157-e54ef40de94c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:44.003836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e91fb693-6bb7-4248-9157-e54ef40de94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.017895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1000/1985] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:44.019953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc28851-9330-4834-8478-a69777426393 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.020238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc28851-9330-4834-8478-a69777426393 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.027640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc28851-9330-4834-8478-a69777426393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.052554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.053661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.054177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.054691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.055746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.058486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.068201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-6cc28851-9330-4834-8478-a69777426393 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:44.069599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6cc28851-9330-4834-8478-a69777426393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.087159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1001/1986] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:44.087159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc28851-9330-4834-8478-a69777426393 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.087159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6cc28851-9330-4834-8478-a69777426393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.105667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 986/1987] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:43 2022] POST /identity/v3/projects => generated 383 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:44.107466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.193818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc28851-9330-4834-8478-a69777426393 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.240212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27133eed-9280-4503-b13a-21f4b8dee0c5 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.249073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.262107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.262107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.262107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.262107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:44.262107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.262107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.279448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 987/1988] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:44.280326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.282326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.303240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.304135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.304450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.304729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.311391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b554469-2bf3-4294-b9f3-52a54d7d64b8 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.323460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.324079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.324433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.324711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.325446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.328928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.334417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.336119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.349174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1002/1989] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:44.350632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.363154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d8bd-3382-4e49-a06f-3101212928c8 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.370335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.372894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.372894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.372894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.373775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.376548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.387391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 988/1990] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:44 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:44.393456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.416487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ecce4-f386-4744-8c88-bf14c724b592 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.422641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1003/1991] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:44 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:44.424453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.426514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.437195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.439052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.439408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.439752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.440478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.446154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.466894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 989/1992] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:44.467975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.482291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021e234b-d63f-43f9-a4d9-9de5fe080d02 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.523930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:44.525482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3e87-81a6-4085-b067-be76d4f684a8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.533105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.537774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.538163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.538721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] PATH_INFO: `/v3/projects/a15ee79400484914b04799525823411e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.540240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a15ee79400484914b04799525823411e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.543409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.567210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.567858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.568206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.568554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.569261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.572238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.585143 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:44.609492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 990/1993] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:44 2022] POST /identity/v3/users => generated 286 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:44.610587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.613213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a15ee79400484914b04799525823411e', 'request_id': 'req-858c5d76-b943-48b1-9330-eb942ce67465'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.620387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a15ee79400484914b04799525823411e', 'request_id': 'req-858c5d76-b943-48b1-9330-eb942ce67465'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.621953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a941f8c8-d263-4c82-8640-27d81dff78fa admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.628307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a15ee79400484914b04799525823411e', 'request_id': 'req-858c5d76-b943-48b1-9330-eb942ce67465'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.641593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] The token cache is being invalidate because project a15ee79400484914b04799525823411e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:44.641913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.642594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.644231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1004/1994] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:44 2022] DELETE /identity/v3/projects/a15ee79400484914b04799525823411e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:44.645014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.646427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.702287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.702983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.703278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.703656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] PATH_INFO: `/v3/projects/0e65a64f689e4479bd69aef258da147a/users/1222871cecdf4463aa040cf828ba9eac/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.725244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e65a64f689e4479bd69aef258da147a, user_id=1222871cecdf4463aa040cf828ba9eac, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.727746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c5d76-b943-48b1-9330-eb942ce67465 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.728282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.732560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.733448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.733769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.734073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.734800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.736548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.744308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 991/1995] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:44 2022] PUT /identity/v3/projects/0e65a64f689e4479bd69aef258da147a/users/1222871cecdf4463aa040cf828ba9eac/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:44.745150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.748927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.761493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1005/1996] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:44.761493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.762428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.779127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab49335-fe95-4f2a-ac8a-e5f256ff3a36 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.842517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead449a-bb9b-4900-9d26-881fbcac3a87 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.850707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.851334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.851619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.851895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.852549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.854492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.859364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.860019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.860300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.860575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] PATH_INFO: `/v3/users/1a23597d83ba4138b824ec4ef671bd3e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.861771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a23597d83ba4138b824ec4ef671bd3e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.863724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.870568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1006/1997] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:44.877231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.900456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04ff09b-ea9a-48a8-ab78-964f948af6c6 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:44.907888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:44.908554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:44.908834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:44.909181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:44.910544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:44.912717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:44.929944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:44.940424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a23597d83ba4138b824ec4ef671bd3e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.941882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a23597d83ba4138b824ec4ef671bd3e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.941995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1007/1998] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:44 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:44.948757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:44.957670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:44.958641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a23597d83ba4138b824ec4ef671bd3e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.960274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a23597d83ba4138b824ec4ef671bd3e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.983848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a23597d83ba4138b824ec4ef671bd3e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:44.986508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53edd6-1272-481b-88d1-6f49057785e6 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.005974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd06c36-3d99-4f9d-a084-ed6e217e780a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.007925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd06c36-3d99-4f9d-a084-ed6e217e780a admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.009533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd06c36-3d99-4f9d-a084-ed6e217e780a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.010889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd06c36-3d99-4f9d-a084-ed6e217e780a admin admin] PATH_INFO: `/v3/projects/0e65a64f689e4479bd69aef258da147a/users/1222871cecdf4463aa040cf828ba9eac/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.015558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd06c36-3d99-4f9d-a084-ed6e217e780a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e65a64f689e4479bd69aef258da147a, user_id=1222871cecdf4463aa040cf828ba9eac, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.028182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd06c36-3d99-4f9d-a084-ed6e217e780a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.054314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 992/1999] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:44 2022] DELETE /identity/v3/users/1a23597d83ba4138b824ec4ef671bd3e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.055590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.056447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1008/2000] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:44 2022] PUT /identity/v3/projects/0e65a64f689e4479bd69aef258da147a/users/1222871cecdf4463aa040cf828ba9eac/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.067399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.068100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.068385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.068657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.110417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] MFA Rules not processed for user `1222871cecdf4463aa040cf828ba9eac`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:45.112203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.126716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:45.214562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1009/2001] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:45.224484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.227144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.233764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e602b6-b505-4a1e-9290-69d7d4a5a1b1 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.243361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42375ea3-c769-4af3-9765-ae27fb666062 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.248384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.249084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.249365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.249702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] PATH_INFO: `/v3/projects/7a7e4d68dee448609fd7e2f8d6c55694` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.250833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7e4d68dee448609fd7e2f8d6c55694)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.251442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.253025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.259185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.259535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.260080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] PATH_INFO: `/v3/projects/82f6c80e9245427aad8acd3c8bd5efbc/users/a621860d8fcb4327820ce6b802e6a725/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.325030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f6c80e9245427aad8acd3c8bd5efbc, user_id=a621860d8fcb4327820ce6b802e6a725, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.328095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.359884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 993/2002] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:45 2022] PUT /identity/v3/projects/82f6c80e9245427aad8acd3c8bd5efbc/users/a621860d8fcb4327820ce6b802e6a725/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.360559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.366198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.382354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7e4d68dee448609fd7e2f8d6c55694', 'request_id': 'req-9342e1ab-441e-4a79-b038-4b2d1c0d269f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.399065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7e4d68dee448609fd7e2f8d6c55694', 'request_id': 'req-9342e1ab-441e-4a79-b038-4b2d1c0d269f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.423998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7e4d68dee448609fd7e2f8d6c55694', 'request_id': 'req-9342e1ab-441e-4a79-b038-4b2d1c0d269f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.451921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] The token cache is being invalidate because project 7a7e4d68dee448609fd7e2f8d6c55694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:45.452297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.456276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9342e1ab-441e-4a79-b038-4b2d1c0d269f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.459387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1010/2003] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/projects/7a7e4d68dee448609fd7e2f8d6c55694 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.462398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.462398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.462398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.470935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9711f70-6c47-46ff-ab97-4688c427f319 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.527350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:45.530339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.583775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.584420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.584704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.584983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.593903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.624820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:45.686444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1011/2004] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:45.687805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.691449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.725326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b95df-bb91-4aaa-9181-bb74b658cb2d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.740436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.740581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.741108 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.741392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.741697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] PATH_INFO: `/v3/users/29069b6f245b465d9a93ba3dc9e86247` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.743218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29069b6f245b465d9a93ba3dc9e86247)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.745158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.750397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.761250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 994/2005] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:45.762266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.764952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.779386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f043585f-54fe-4471-b017-1ae9255ff4a8 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:45.784644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:45.785432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.785765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.786040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] PATH_INFO: `/v3/projects/9dba3ab8d57344c48e7df7f3a1ec339d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:45.787247 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dba3ab8d57344c48e7df7f3a1ec339d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:45.789548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:45.810965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:45.854799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29069b6f245b465d9a93ba3dc9e86247'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.855226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29069b6f245b465d9a93ba3dc9e86247'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.874480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29069b6f245b465d9a93ba3dc9e86247'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.877046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29069b6f245b465d9a93ba3dc9e86247'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.907959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29069b6f245b465d9a93ba3dc9e86247'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.918352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dba3ab8d57344c48e7df7f3a1ec339d', 'request_id': 'req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.924466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dba3ab8d57344c48e7df7f3a1ec339d', 'request_id': 'req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.924678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1012/2006] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/users/29069b6f245b465d9a93ba3dc9e86247 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.925662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:45.927004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:45.935946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dba3ab8d57344c48e7df7f3a1ec339d', 'request_id': 'req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.965948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] The token cache is being invalidate because project 9dba3ab8d57344c48e7df7f3a1ec339d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:45.967259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.969912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c9336f8b-badb-484b-b9e9-cdc0ffb0d43e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:45.974271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 995/2007] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:45 2022] DELETE /identity/v3/projects/9dba3ab8d57344c48e7df7f3a1ec339d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:45.979677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5c93-d773-4125-94c0-e3e411c3ac6a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:45.980140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5c93-d773-4125-94c0-e3e411c3ac6a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:45.980957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a5c93-d773-4125-94c0-e3e411c3ac6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.029490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-611a5c93-d773-4125-94c0-e3e411c3ac6a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:46.032182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-611a5c93-d773-4125-94c0-e3e411c3ac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.164833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 996/2008] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:46.166946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.167447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.168007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.169543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bcb7a6-9a34-4aab-a236-8f59ae5d5197 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.182756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.183411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.183693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.183971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.193491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:46.195025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.203043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.220393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 997/2009] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:46.221703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.236933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.245606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac33d1-6093-468b-8a32-d61a81cd82ef None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.258936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.259652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.260002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.260392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.261164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:46.263216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.264740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.281604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.318526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1013/2010] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:46.319296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 998/2011] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:46.324564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.349631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb63c93a-8a0e-4f6f-bdd1-2511b6830e65 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.382345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.383095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.383812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.383812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.384438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.386627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.487756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1014/2012] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/projects => generated 367 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:46.492691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.521513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba2a29f-abda-40db-9745-f8d25e540884 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.545992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:46.648542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.649383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.649842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.650235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.664945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.668292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.721602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 999/2013] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/users => generated 284 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:46.725517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.736721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fd9bb-83a1-4828-8f3f-049bb8ef422a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.743161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.744003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.744181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.744464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.745490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.747456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.759358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1015/2014] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:46.763897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.778051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4b141-d77b-4559-90c7-8469c8a72999 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.782440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.784237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:46.789133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.790268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.792355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.792839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.793778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.795698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.802526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfcce8-0f0c-452e-91cf-5a5e5c67816c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.811149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1000/2015] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:46.815658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:46.819194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.820134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.820616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.821063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] PATH_INFO: `/v3/projects/cc753a070be64b3c807f848146433c14` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.823126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc753a070be64b3c807f848146433c14)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.825884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.833673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b49c59-d979-4bbd-a1ce-d5f938c68522 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:46.842014 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e645317-1dff-4752-93b9-9dc1c6a2f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:46.842915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e645317-1dff-4752-93b9-9dc1c6a2f690 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.843334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e645317-1dff-4752-93b9-9dc1c6a2f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.844544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e645317-1dff-4752-93b9-9dc1c6a2f690 admin admin] PATH_INFO: `/v3/projects/79ceefe925a54c41ba4e2a8ff3bf52b4/users/70226ea320c548829d6d73be6db12561/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.882044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e645317-1dff-4752-93b9-9dc1c6a2f690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ceefe925a54c41ba4e2a8ff3bf52b4, user_id=70226ea320c548829d6d73be6db12561, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:46.885803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e645317-1dff-4752-93b9-9dc1c6a2f690 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:46.908869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc753a070be64b3c807f848146433c14', 'request_id': 'req-3330efcd-2d2c-482e-8edd-88e94a4767b5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.915612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc753a070be64b3c807f848146433c14', 'request_id': 'req-3330efcd-2d2c-482e-8edd-88e94a4767b5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.927284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc753a070be64b3c807f848146433c14', 'request_id': 'req-3330efcd-2d2c-482e-8edd-88e94a4767b5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.941911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1001/2016] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:46 2022] PUT /identity/v3/projects/79ceefe925a54c41ba4e2a8ff3bf52b4/users/70226ea320c548829d6d73be6db12561/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:46.943028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:46.945445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d00bc-572d-403f-8553-61a487d3610c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.945772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d00bc-572d-403f-8553-61a487d3610c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.946027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] The token cache is being invalidate because project cc753a070be64b3c807f848146433c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:46.946491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.947458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d00bc-572d-403f-8553-61a487d3610c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.948371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3330efcd-2d2c-482e-8edd-88e94a4767b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:46.955367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1016/2017] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:46 2022] DELETE /identity/v3/projects/cc753a070be64b3c807f848146433c14 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:46.957437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa12a35-f946-4afc-b7bc-68b1e1e4c5b2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:46.958040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa12a35-f946-4afc-b7bc-68b1e1e4c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:46.958377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa12a35-f946-4afc-b7bc-68b1e1e4c5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:46.993868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-850d00bc-572d-403f-8553-61a487d3610c None None] MFA Rules not processed for user `70226ea320c548829d6d73be6db12561`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:46.995398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-850d00bc-572d-403f-8553-61a487d3610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.009192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5aa12a35-f946-4afc-b7bc-68b1e1e4c5b2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:47.024726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5aa12a35-f946-4afc-b7bc-68b1e1e4c5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.102496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1017/2018] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:47.113343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.113681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.113959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.156533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:47.157795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.165527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1002/2019] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:47.170100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d00bc-572d-403f-8553-61a487d3610c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.172608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-850d00bc-572d-403f-8553-61a487d3610c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.186078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1018/2020] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:47.189963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.197148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d00bc-572d-403f-8553-61a487d3610c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.205813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.206457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.206723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.206996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.207639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.210485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.211155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56ac05-3cfe-4014-bf8b-49c26cb3e92f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.220830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.221459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.221785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.222056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.222715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:47.223017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.226942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.262110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1003/2021] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:47.276686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1019/2022] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:47.277385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.293326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.309399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62ed6d-70f6-4b01-acff-3925eb08bd74 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.345703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:47.382353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231db1f5-54e4-4e63-8727-ecf41067b3e2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.390701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.391486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.391832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.392172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.392940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.395911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.437680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1004/2023] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:47.443550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.447378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.448026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.448284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.448562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.449194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.453413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.458513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef826fe-ace7-400d-91ac-f66c3fb5fcce admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.464154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.464904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.465285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.465703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.467001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.468828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.495428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1020/2024] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/projects => generated 365 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:47.495701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1005/2025] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:47.500996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.502322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.523627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf5fe3-6d62-45c1-9a99-fd34a43e7718 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.589508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4767ed7-da25-4a54-90f4-70587492ee39 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.595267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.596119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.596873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.597123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.597901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.600353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.624205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:47.626987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd86a44a-6d17-435c-82b5-416f0b51169e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.627796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86a44a-6d17-435c-82b5-416f0b51169e admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.628255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86a44a-6d17-435c-82b5-416f0b51169e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.628564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd86a44a-6d17-435c-82b5-416f0b51169e admin admin] PATH_INFO: `/v3/projects/bde60308fadd4d16a297eab9c8a9e25c/users/571cb54db3934087b3c0132cd643cda5/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.651643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1021/2026] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:47.652564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.678210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86a44a-6d17-435c-82b5-416f0b51169e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde60308fadd4d16a297eab9c8a9e25c, user_id=571cb54db3934087b3c0132cd643cda5, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.682678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd86a44a-6d17-435c-82b5-416f0b51169e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.690991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e5c31-98da-4d6a-8f2a-b7cae866bf40 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.710703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.711341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.711627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.711926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.712586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.714230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.717049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1006/2027] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:47 2022] PUT /identity/v3/projects/bde60308fadd4d16a297eab9c8a9e25c/users/571cb54db3934087b3c0132cd643cda5/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:47.718690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.718987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.719456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.744362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1022/2028] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:47.745421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.757280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] MFA Rules not processed for user `a621860d8fcb4327820ce6b802e6a725`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:47.758548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.767418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9495c77f-8f5f-48b1-ab6d-48f9eb92ca3b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.902584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.904666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.905599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.906347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.907365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.911755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.935206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1007/2029] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:47.940417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.952171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1023/2030] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:47.953556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.956260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:47.956572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87266eb9-9594-4c70-89a7-c3f9f9d6b2fa None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:47.961876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:47.962590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:47.963017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:47.963304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:47.964109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:47.966311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:47.984419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1008/2031] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:47.985501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:47.988299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.012286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:48.066204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341208af-00aa-4cd6-a339-ee76af1546aa admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.070258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.070887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.071334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.071614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.073089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43ba0a-fe39-40c9-81a5-da8edb734806 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.073439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.077204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.077855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.078420 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.078798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.089263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.091273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.102594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1009/2032] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:48.102594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.119220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6822bf7-32ad-4fbf-8373-6e992d7ac661 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.124397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.125154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.125763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.126253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] PATH_INFO: `/v3/projects/bde60308fadd4d16a297eab9c8a9e25c/users/571cb54db3934087b3c0132cd643cda5/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.128343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde60308fadd4d16a297eab9c8a9e25c, user_id=571cb54db3934087b3c0132cd643cda5, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.131149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.144270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1010/2033] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:48 2022] PUT /identity/v3/projects/bde60308fadd4d16a297eab9c8a9e25c/users/571cb54db3934087b3c0132cd643cda5/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:48.164600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.174264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.177518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bd2c3-1dd1-4c56-999b-96d1eff100b4 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.201705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1024/2034] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:48.207334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.207567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.207878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.252242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e7599-64dc-41f9-b413-37f86a86a395 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.263921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b606721-830a-4b26-bf0e-9e0616af2360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.264789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b606721-830a-4b26-bf0e-9e0616af2360 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.265842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b606721-830a-4b26-bf0e-9e0616af2360 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.287874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b606721-830a-4b26-bf0e-9e0616af2360 admin admin] PATH_INFO: `/v3/projects/92651e9bbc2d4585b617d70ee046d693/users/0292fbf290bd4c01891e8126e2ae0049/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.326610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] MFA Rules not processed for user `571cb54db3934087b3c0132cd643cda5`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:48.328806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b606721-830a-4b26-bf0e-9e0616af2360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92651e9bbc2d4585b617d70ee046d693, user_id=0292fbf290bd4c01891e8126e2ae0049, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.330809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.333054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b606721-830a-4b26-bf0e-9e0616af2360 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.374655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:48.374882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1011/2035] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:48 2022] PUT /identity/v3/projects/92651e9bbc2d4585b617d70ee046d693/users/0292fbf290bd4c01891e8126e2ae0049/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:48.376334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.377198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.377198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.452587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] MFA Rules not processed for user `0292fbf290bd4c01891e8126e2ae0049`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:48.453854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:48.496156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1025/2036] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:48 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:48.572939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.691245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1012/2037] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:48.697140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:48.734584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78dc8fe-939e-4329-9f89-1c47253d7175 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.746727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:48.822840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.824806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.824806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.824806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.899596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7294a607-cf7f-4700-af25-b93a10c211b2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:48.920430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f1085f-1647-4eca-80c2-65c7d3703135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:48.925755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1085f-1647-4eca-80c2-65c7d3703135 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:48.926446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1085f-1647-4eca-80c2-65c7d3703135 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:48.926847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.927026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f1085f-1647-4eca-80c2-65c7d3703135 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:48.928116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1085f-1647-4eca-80c2-65c7d3703135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:48.928448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.939099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f1085f-1647-4eca-80c2-65c7d3703135 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:48.942444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1026/2038] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:49.010670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1013/2039] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:48 2022] POST /identity/v3/projects => generated 363 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:49.015783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.032287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a129e-7b1a-4545-90c7-19b68ec60c22 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.061468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.141863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.143589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.143962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.144954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.147188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.152257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.221268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.221268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.221268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.243401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1027/2040] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:49 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:49.244865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.287156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9677a3-d581-4fd7-9c76-afff7145e00c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.290446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] MFA Rules not processed for user `a621860d8fcb4327820ce6b802e6a725`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:49.296631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.303939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.305307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.306404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.307057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.308668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.328471 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.375346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1028/2041] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:49.375346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.437805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1014/2042] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:49.438690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.474888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364a169f-c7df-4715-b957-5e81453ff3a2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.489656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.492465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.493878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.494968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] PATH_INFO: `/v3/projects/8c750b23ea9e4363a1e7619941fca616/users/840bfdc5bd764d08bac15ae812e6359d/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.539688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a8c75-4bd5-4a1b-a1eb-5d639e16d583 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.545245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.546283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.546788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.547204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.562537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c750b23ea9e4363a1e7619941fca616, user_id=840bfdc5bd764d08bac15ae812e6359d, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.565683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.590576 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1015/2043] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:49 2022] PUT /identity/v3/projects/8c750b23ea9e4363a1e7619941fca616/users/840bfdc5bd764d08bac15ae812e6359d/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:49.594415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.695532 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.697294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.721516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1029/2044] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:49.723844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8deea7b-27d4-48f2-a281-27c046118392 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.730960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.732436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.733055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.733932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.735018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.736527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.737110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.755773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1016/2045] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:49.757954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:49.764993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a304442-bd14-42a8-bf60-edeffa7ade21 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.773525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dc8ad2-c446-491d-9f3f-838425522f20 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:49.780829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a311e-24cb-4941-8e03-02c86ecec117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.795631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a311e-24cb-4941-8e03-02c86ecec117 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.796152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a311e-24cb-4941-8e03-02c86ecec117 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.796642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a311e-24cb-4941-8e03-02c86ecec117 admin admin] PATH_INFO: `/v3/projects/8c750b23ea9e4363a1e7619941fca616/users/840bfdc5bd764d08bac15ae812e6359d/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.801060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a311e-24cb-4941-8e03-02c86ecec117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c750b23ea9e4363a1e7619941fca616, user_id=840bfdc5bd764d08bac15ae812e6359d, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:49.804486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a311e-24cb-4941-8e03-02c86ecec117 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:49.829386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1017/2046] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:49 2022] PUT /identity/v3/projects/8c750b23ea9e4363a1e7619941fca616/users/840bfdc5bd764d08bac15ae812e6359d/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:49.832082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.832708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.833120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.855002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.882066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] MFA Rules not processed for user `840bfdc5bd764d08bac15ae812e6359d`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:49.883921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.921711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:49.922998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:49.924811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:49.925911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:49.931320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:49.957664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:49.996550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1018/2047] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:50.071437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.073047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.094538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1030/2048] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:50.178431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:50.261230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e4dcd-3fc9-4cfb-b6cc-d7d1447e0cfa None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:50.265115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:50.271111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.271111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.271111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.284475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.286084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.313490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1019/2049] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:50.499255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:50.511627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc7106b-6d90-434a-a708-3dd6314b5a13 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:50.520481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b2975-2406-4bfc-baad-f5d8d0262313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:50.521334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b2975-2406-4bfc-baad-f5d8d0262313 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:50.521814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b2975-2406-4bfc-baad-f5d8d0262313 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:50.521993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b2975-2406-4bfc-baad-f5d8d0262313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:50.650597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b2975-2406-4bfc-baad-f5d8d0262313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:50.655247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8b2975-2406-4bfc-baad-f5d8d0262313 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:50.687927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1031/2050] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:53.710568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:53.787663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7556cab-6ecc-4325-9665-b60b95158d31 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:53.794475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:53.795436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.795772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.796048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] PATH_INFO: `/v3/users/0292fbf290bd4c01891e8126e2ae0049` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:53.799045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0292fbf290bd4c01891e8126e2ae0049)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:53.801506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:53.863382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:53.880528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0292fbf290bd4c01891e8126e2ae0049'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:53.881316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0292fbf290bd4c01891e8126e2ae0049'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:53.890997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0292fbf290bd4c01891e8126e2ae0049'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:53.892136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0292fbf290bd4c01891e8126e2ae0049'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:53.925665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0292fbf290bd4c01891e8126e2ae0049'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:53.963975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1020/2051] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:53 2022] DELETE /identity/v3/users/0292fbf290bd4c01891e8126e2ae0049 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:53.968612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:53.969599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:53.970249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.003074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:54.004324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.129273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1032/2052] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:54.140217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:54.141752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.240069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e47674-77e2-45c1-9b9d-6d73dfe80a1e admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:54.246473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:54.247351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:54.247703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:54.248067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.261892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:54.266893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:54.304613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1021/2053] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:54.672285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:54.674004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.687506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd238f-2c3b-4558-999e-3aad1ac4a4d8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:54.692131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:54.692772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:54.693395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:54.693395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] PATH_INFO: `/v3/users/466563f90b0d46d69dbe2038c11d5679` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.694663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] RBAC: Authorizing `identity:delete_user(user_id=466563f90b0d46d69dbe2038c11d5679)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:54.696629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:54.742426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:54.744308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.745948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:54.753279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466563f90b0d46d69dbe2038c11d5679'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.754144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466563f90b0d46d69dbe2038c11d5679'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.763568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553bd71-c7e5-4e78-9ee7-6a856b1ee434 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:54.766807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466563f90b0d46d69dbe2038c11d5679'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.767823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466563f90b0d46d69dbe2038c11d5679'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.775221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:54.775891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:54.776377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:54.776616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] PATH_INFO: `/v3/projects/92651e9bbc2d4585b617d70ee046d693` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.778636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92651e9bbc2d4585b617d70ee046d693)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:54.780924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:54.786711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c953a0f0-3c8b-4d64-a708-2a477757e137 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466563f90b0d46d69dbe2038c11d5679'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.807390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1033/2054] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:54 2022] DELETE /identity/v3/users/466563f90b0d46d69dbe2038c11d5679 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:54.812327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:54.812618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:54.812894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.833955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:54.835178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.853196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92651e9bbc2d4585b617d70ee046d693', 'request_id': 'req-56022104-14f1-4474-bb57-6b2d5a27f728'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.860415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92651e9bbc2d4585b617d70ee046d693', 'request_id': 'req-56022104-14f1-4474-bb57-6b2d5a27f728'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.867496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92651e9bbc2d4585b617d70ee046d693', 'request_id': 'req-56022104-14f1-4474-bb57-6b2d5a27f728'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.881828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] The token cache is being invalidate because project 92651e9bbc2d4585b617d70ee046d693 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:54.882116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.882739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:54.884263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1022/2055] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:54 2022] DELETE /identity/v3/projects/92651e9bbc2d4585b617d70ee046d693 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:54.889042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:54.890416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.902859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1034/2056] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:54.912613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:54.914043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:54.934111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:54.957385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56022104-14f1-4474-bb57-6b2d5a27f728 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:54.962024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:54.962646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:54.962921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:54.963190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] PATH_INFO: `/v3/users/840bfdc5bd764d08bac15ae812e6359d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:54.964322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] RBAC: Authorizing `identity:delete_user(user_id=840bfdc5bd764d08bac15ae812e6359d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:54.966455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:54.972689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1a5b41-0cca-4ba3-a3f1-63b8ce3b20d8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:55.015647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '840bfdc5bd764d08bac15ae812e6359d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.016523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '840bfdc5bd764d08bac15ae812e6359d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.022390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:55.023008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.023292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.023499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '840bfdc5bd764d08bac15ae812e6359d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.023596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.024235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '840bfdc5bd764d08bac15ae812e6359d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.025429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.039094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:55.041042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:55.042537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '840bfdc5bd764d08bac15ae812e6359d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.052227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1035/2057] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:46:55.063636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1023/2058] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:54 2022] DELETE /identity/v3/users/840bfdc5bd764d08bac15ae812e6359d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:55.540067 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:55.543792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.594854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ce2d0-d96d-4f7c-bde6-9a79f054275c service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:55.621253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:55.685702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:55.687334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.695496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:55.696270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.696673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.697063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] PATH_INFO: `/v3/projects/8c750b23ea9e4363a1e7619941fca616` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.698379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c750b23ea9e4363a1e7619941fca616)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:55.701549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:55.805419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184be62-af5f-44d5-b24f-e1e1402e743a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:55.822102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c750b23ea9e4363a1e7619941fca616', 'request_id': 'req-863364fc-82b7-4fb3-bf75-56d88150d501'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.846357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c750b23ea9e4363a1e7619941fca616', 'request_id': 'req-863364fc-82b7-4fb3-bf75-56d88150d501'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.864155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c750b23ea9e4363a1e7619941fca616', 'request_id': 'req-863364fc-82b7-4fb3-bf75-56d88150d501'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.893967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] The token cache is being invalidate because project 8c750b23ea9e4363a1e7619941fca616 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:55.893967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.893967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-863364fc-82b7-4fb3-bf75-56d88150d501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.896478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1036/2059] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:55 2022] DELETE /identity/v3/projects/8c750b23ea9e4363a1e7619941fca616 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:55.902284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22ad68-2c6c-43f6-b3fd-e17ec1f1980f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.902794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22ad68-2c6c-43f6-b3fd-e17ec1f1980f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.904354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f22ad68-2c6c-43f6-b3fd-e17ec1f1980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.923302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:55.924020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.925767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.926188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] PATH_INFO: `/v3/users/70226ea320c548829d6d73be6db12561` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.927473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] RBAC: Authorizing `identity:delete_user(user_id=70226ea320c548829d6d73be6db12561)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:55.929669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:55.935038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1f22ad68-2c6c-43f6-b3fd-e17ec1f1980f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:55.937287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1f22ad68-2c6c-43f6-b3fd-e17ec1f1980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.938534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:55.951549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1037/2060] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:55.959815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:55.960104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:55.960364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:55.984463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:55.985716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:55.993539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70226ea320c548829d6d73be6db12561'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.997790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70226ea320c548829d6d73be6db12561'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:55.997935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1038/2061] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:55.999295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.004577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70226ea320c548829d6d73be6db12561'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.005308 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70226ea320c548829d6d73be6db12561'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.008882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d077a-ca6e-4161-85ec-86677d2497b3 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.014973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.015172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3d92c315-e209-449a-9fa1-a0b4d30c531b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70226ea320c548829d6d73be6db12561'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.015625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.016261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.016553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.017282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:56.017595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.019257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.034203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1039/2062] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:46:56.038556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.040269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1024/2063] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:55 2022] DELETE /identity/v3/users/70226ea320c548829d6d73be6db12561 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:56.040406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.044845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.045130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.045403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.056555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97165277-8f2e-4cd8-a37c-421d9be93ee8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.068531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:56.070195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.126940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.127659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.128006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.128353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.129090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.131253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.148822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1025/2064] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:56.160289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.164685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.170100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1040/2065] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:56 2022] POST /identity/v3/projects => generated 387 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:56.174341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.184672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:56.233510 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2171ac-1276-40d1-bf58-104ce7b10a86 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.251835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578e653-8914-4533-84a2-4c0b8dd1166f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.268429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.270248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.270543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.270818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.271461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.273495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.287654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:56.301784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.303309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1041/2066] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:56 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:56.306465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.307093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.307477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.307758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.320146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.321733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.324390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b199-bd55-4c97-b1f0-0592c236cb07 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.330857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.332257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.332664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.332947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.333543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1026/2067] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:56.333808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.335679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.352591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.353171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1042/2068] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:56.363635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.365965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29355a80-7454-4524-b62b-654614fd5f72 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.367529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.373958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.374675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.375362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.375362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.376084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.378858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.390290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581493f-e317-4bc7-8ea4-ecb5819b4e25 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.391506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1027/2069] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:46:56.396017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.397976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.398628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.398969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.399190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] PATH_INFO: `/v3/projects/9d506ccfc1464224912d2b550f40ef09` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.401730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d506ccfc1464224912d2b550f40ef09)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.405156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.411607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71294398-70aa-4460-80dd-f5c03ee06946 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.418383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca30f1d-e067-4adb-92a1-06765ee783d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.419965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30f1d-e067-4adb-92a1-06765ee783d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.420160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30f1d-e067-4adb-92a1-06765ee783d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.420862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca30f1d-e067-4adb-92a1-06765ee783d4 admin admin] PATH_INFO: `/v3/projects/ad1f06dea5c24d24addbd736669011d5/users/68b5753c6b2c4e6da9fcd08467af7f30/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.444749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca30f1d-e067-4adb-92a1-06765ee783d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1f06dea5c24d24addbd736669011d5, user_id=68b5753c6b2c4e6da9fcd08467af7f30, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.448814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca30f1d-e067-4adb-92a1-06765ee783d4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.496284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1028/2070] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:56 2022] PUT /identity/v3/projects/ad1f06dea5c24d24addbd736669011d5/users/68b5753c6b2c4e6da9fcd08467af7f30/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:56.505076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d506ccfc1464224912d2b550f40ef09', 'request_id': 'req-3147de85-8880-467e-9c8c-d4fb14b4901a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.509004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.509287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.509594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.514536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d506ccfc1464224912d2b550f40ef09', 'request_id': 'req-3147de85-8880-467e-9c8c-d4fb14b4901a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.526320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d506ccfc1464224912d2b550f40ef09', 'request_id': 'req-3147de85-8880-467e-9c8c-d4fb14b4901a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.546338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] The token cache is being invalidate because project 9d506ccfc1464224912d2b550f40ef09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:56.546700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.547501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.550338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1043/2071] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:56 2022] DELETE /identity/v3/projects/9d506ccfc1464224912d2b550f40ef09 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:56.554966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.563794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] MFA Rules not processed for user `68b5753c6b2c4e6da9fcd08467af7f30`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:56.567527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.570691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.663268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147de85-8880-467e-9c8c-d4fb14b4901a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.669311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.670192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.670595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.670878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] PATH_INFO: `/v3/users/bb862a36f1cf4a9880c60134d77c934f` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.672069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb862a36f1cf4a9880c60134d77c934f)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.674356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.679487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1029/2072] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:56 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:56.684049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.692920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.706654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:56.709411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbae1f-380f-4052-b087-ba933d7b5553 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.715541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.716324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.716783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.717283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.718117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.721547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.768246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1030/2073] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:56 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:46:56.769916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.773000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.779709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb862a36f1cf4a9880c60134d77c934f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.780708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb862a36f1cf4a9880c60134d77c934f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.792195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb862a36f1cf4a9880c60134d77c934f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.793614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb862a36f1cf4a9880c60134d77c934f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.815830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb862a36f1cf4a9880c60134d77c934f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:56.831192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1044/2074] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:56 2022] DELETE /identity/v3/users/bb862a36f1cf4a9880c60134d77c934f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:56.831950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:56.836893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:56.880970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404b1c76-0c69-452e-9aa1-13207162cb8f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.915534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:56.975850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60770fb8-ce43-47ff-982b-672d1595569d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:56.978073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.978782 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.979121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.979400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] PATH_INFO: `/v3/projects/79ceefe925a54c41ba4e2a8ff3bf52b4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.981919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:56.982279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] RBAC: Authorizing `identity:delete_project(project_id=79ceefe925a54c41ba4e2a8ff3bf52b4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.982957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:56.983646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:56.984093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:56.984871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:56.986259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:56.987586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.021629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1045/2075] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:56 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:57.023378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.036605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603de2-350d-46fd-a618-897ce61a10ef admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.042608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.043382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.043765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.044206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.045262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.047592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.066161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1046/2076] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:57.081260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.093327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79ceefe925a54c41ba4e2a8ff3bf52b4', 'request_id': 'req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.094998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c1b048-fbbb-4085-8d8d-08bfe9c8556b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.106838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79ceefe925a54c41ba4e2a8ff3bf52b4', 'request_id': 'req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.119808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79ceefe925a54c41ba4e2a8ff3bf52b4', 'request_id': 'req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.147455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] The token cache is being invalidate because project 79ceefe925a54c41ba4e2a8ff3bf52b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:57.147860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.148690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.151979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1031/2077] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:56 2022] DELETE /identity/v3/projects/79ceefe925a54c41ba4e2a8ff3bf52b4 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:57.156821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.158970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.213937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.214977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.215307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.221419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] PATH_INFO: `/v3/projects/681e577932364d41b73ea7dc684f8024/users/ff77b9d2509e4ffaa8e830c7d75e364d/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.247875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:57.252425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06f2a7-b11d-4347-ab39-b49b8a9c047d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.258292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681e577932364d41b73ea7dc684f8024, user_id=ff77b9d2509e4ffaa8e830c7d75e364d, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.258659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.259961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.259961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.259961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] PATH_INFO: `/v3/users/571cb54db3934087b3c0132cd643cda5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.261136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=571cb54db3934087b3c0132cd643cda5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.263701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.281547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.312661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1047/2078] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:57 2022] PUT /identity/v3/projects/681e577932364d41b73ea7dc684f8024/users/ff77b9d2509e4ffaa8e830c7d75e364d/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:57.317503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.319263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.329758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '571cb54db3934087b3c0132cd643cda5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.330520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '571cb54db3934087b3c0132cd643cda5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.342004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '571cb54db3934087b3c0132cd643cda5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.342820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '571cb54db3934087b3c0132cd643cda5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.344677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:57.353721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '571cb54db3934087b3c0132cd643cda5'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.378314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1032/2079] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:57 2022] DELETE /identity/v3/users/571cb54db3934087b3c0132cd643cda5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:57.379555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.381372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.417414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08054f7f-f0b2-40c4-9bad-432b089df625 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.457170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7899e-9b9f-4bed-a6a2-21efea1e1613 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.463863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.465057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.465360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.466541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] PATH_INFO: `/v3/projects/f26172d8328f4be493537cb3ce2ad2d5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.467509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26172d8328f4be493537cb3ce2ad2d5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.469882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.494041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.494648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.494983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.495263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.496025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.498636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.561752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1048/2080] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:57.562996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.565235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.593935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8387154-eaed-4837-96cb-213597e6f650 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.595004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26172d8328f4be493537cb3ce2ad2d5', 'request_id': 'req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.610193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26172d8328f4be493537cb3ce2ad2d5', 'request_id': 'req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.622384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26172d8328f4be493537cb3ce2ad2d5', 'request_id': 'req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.658131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] The token cache is being invalidate because project f26172d8328f4be493537cb3ce2ad2d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:57.658131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.659473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.663138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1033/2081] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:57 2022] DELETE /identity/v3/projects/f26172d8328f4be493537cb3ce2ad2d5 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:57.667924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.670226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.736785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fca860-b677-4300-bfe1-2068c5e3f305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.738023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca860-b677-4300-bfe1-2068c5e3f305 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.738023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca860-b677-4300-bfe1-2068c5e3f305 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.738654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fca860-b677-4300-bfe1-2068c5e3f305 admin admin] PATH_INFO: `/v3/projects/681e577932364d41b73ea7dc684f8024/users/ff77b9d2509e4ffaa8e830c7d75e364d/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.741764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fca860-b677-4300-bfe1-2068c5e3f305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=681e577932364d41b73ea7dc684f8024, user_id=ff77b9d2509e4ffaa8e830c7d75e364d, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.745299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fca860-b677-4300-bfe1-2068c5e3f305 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.760562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1049/2082] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:57 2022] PUT /identity/v3/projects/681e577932364d41b73ea7dc684f8024/users/ff77b9d2509e4ffaa8e830c7d75e364d/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:57.771894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.772224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.772503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.800639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b67aea1-84a5-4e0c-a1b3-d48364ee709b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:57.825216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:57.843337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] MFA Rules not processed for user `ff77b9d2509e4ffaa8e830c7d75e364d`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:57.845226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.865275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:57.887217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:57.888555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:57.888933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:57.889305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] PATH_INFO: `/v3/users/1222871cecdf4463aa040cf828ba9eac` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:57.891342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1222871cecdf4463aa040cf828ba9eac)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:57.894834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:57.961759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1222871cecdf4463aa040cf828ba9eac'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.962530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1222871cecdf4463aa040cf828ba9eac'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.970005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1222871cecdf4463aa040cf828ba9eac'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.970679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1222871cecdf4463aa040cf828ba9eac'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:57.983037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1050/2083] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:57 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:57.983037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:57.983037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:57.983648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1222871cecdf4463aa040cf828ba9eac'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.001909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4862a97-a982-4595-b2c5-c38afd2dbc9a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.004037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1034/2084] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:57 2022] DELETE /identity/v3/users/1222871cecdf4463aa040cf828ba9eac => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:46:58.007911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.010375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.115876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f610400d-5d51-498a-bf37-3d47604dad87 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.116678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.117964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.119380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.119759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] PATH_INFO: `/v3/projects/bde60308fadd4d16a297eab9c8a9e25c` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.120167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.120793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.121060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.121330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.123155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde60308fadd4d16a297eab9c8a9e25c)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.123808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.126892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.219972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:58.252193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde60308fadd4d16a297eab9c8a9e25c', 'request_id': 'req-cf558be4-4126-422c-95a6-d43a82a1d411'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.260634 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde60308fadd4d16a297eab9c8a9e25c', 'request_id': 'req-cf558be4-4126-422c-95a6-d43a82a1d411'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.276610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde60308fadd4d16a297eab9c8a9e25c', 'request_id': 'req-cf558be4-4126-422c-95a6-d43a82a1d411'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.280937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.282570 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f68bd5-4fec-48b7-ab61-106568415caf service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.293977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] The token cache is being invalidate because project bde60308fadd4d16a297eab9c8a9e25c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:58.294520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.295431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.298998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1035/2086] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:58.301433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1051/2086] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:57 2022] DELETE /identity/v3/projects/bde60308fadd4d16a297eab9c8a9e25c => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:58.302255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.302728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.303007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.363872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.381390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.414498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.415970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.502549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1036/2087] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:58.503651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.504852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.509218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:58.534812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5063984b-ac74-4de5-b251-500f75bda8d9 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.540869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.541773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.542116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.542455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] PATH_INFO: `/v3/projects/0e65a64f689e4479bd69aef258da147a` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.544278 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e65a64f689e4479bd69aef258da147a)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.546353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.549822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558be4-4126-422c-95a6-d43a82a1d411 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.554738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.555375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.555651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.556036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] PATH_INFO: `/v3/users/68b5753c6b2c4e6da9fcd08467af7f30` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.557167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b5753c6b2c4e6da9fcd08467af7f30)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.559770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.602800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e65a64f689e4479bd69aef258da147a', 'request_id': 'req-1099665f-f9c6-4bff-9140-e253b7bde2cb'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.610259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e65a64f689e4479bd69aef258da147a', 'request_id': 'req-1099665f-f9c6-4bff-9140-e253b7bde2cb'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.615751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b5753c6b2c4e6da9fcd08467af7f30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.616414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b5753c6b2c4e6da9fcd08467af7f30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.616543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e65a64f689e4479bd69aef258da147a', 'request_id': 'req-1099665f-f9c6-4bff-9140-e253b7bde2cb'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.624937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b5753c6b2c4e6da9fcd08467af7f30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.625808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b5753c6b2c4e6da9fcd08467af7f30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.633731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] The token cache is being invalidate because project 0e65a64f689e4479bd69aef258da147a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:58.634234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.634883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1099665f-f9c6-4bff-9140-e253b7bde2cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.637437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1037/2088] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:58 2022] DELETE /identity/v3/projects/0e65a64f689e4479bd69aef258da147a => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:58.643478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-57d3eeb7-f93c-4dfa-8484-6ff55279277f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b5753c6b2c4e6da9fcd08467af7f30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:58.643624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4524-e892-4c58-baa5-ae3a7fbf34ca None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.643846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4524-e892-4c58-baa5-ae3a7fbf34ca None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.644118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb4524-e892-4c58-baa5-ae3a7fbf34ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.670714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-97eb4524-e892-4c58-baa5-ae3a7fbf34ca None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.672072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-97eb4524-e892-4c58-baa5-ae3a7fbf34ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.676448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1052/2089] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:58 2022] DELETE /identity/v3/users/68b5753c6b2c4e6da9fcd08467af7f30 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:58.678435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.679042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.680206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.705995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.707363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.755894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1038/2090] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:58.757624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc2ff-7c6b-47ac-bfae-82e08a30cba7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.757624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc2ff-7c6b-47ac-bfae-82e08a30cba7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.757837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecdc2ff-7c6b-47ac-bfae-82e08a30cba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.768080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:58.781068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-9ecdc2ff-7c6b-47ac-bfae-82e08a30cba7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.782388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9ecdc2ff-7c6b-47ac-bfae-82e08a30cba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.784944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1053/2091] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:58.785989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.797954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d00f3a-8a64-4b5b-9495-c8f0c7ddd23b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.799685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1039/2092] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:58.801833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.801833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.801833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.808179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.813704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.814130 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.814613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.815417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:58.816149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.818372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.829171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:58.830512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.834060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1054/2093] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:58.835141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.837527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:58.844394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1040/2094] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:58.845352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.857298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a85fe-c01c-4a70-8103-9b0afbfda1ed None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.863291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.864403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.864689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.864965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.866284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:58.868875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:58.905761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1041/2095] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:58 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:58.906608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:58.912709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:58.917769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf25302-77c2-45ae-89e7-07d1e12eeac6 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:58.990124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:58.990994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:58.991346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:58.991689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:58.998159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0114a-874c-4b1a-be98-8acc3325d8ab admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.003487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.003668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.004307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.004655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.005044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.005283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.005985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:46:59.006375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.008424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.016029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1055/2096] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:46:59.016638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.025049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1042/2097] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:46:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:46:59.026505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71a60e-7690-43b6-af6b-4eda9f57819f service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.030640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.033521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.034145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.034423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.034694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.035434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.037355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.047914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baea51a-80e3-4170-8e06-78a1a77ac0ac admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.057808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.059732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.060386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.061181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.062798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.066446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.067549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1056/2098] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/users => generated 292 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.068454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.080993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d12dd79-6f77-4240-ba18-8fc2b70e9097 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.088372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.089173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.091081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.091424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.092072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.094423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.113732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1057/2099] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.116384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.122232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1043/2100] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.123886 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.129008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3e73c-9d40-4e19-866d-2ea8c0ee0aed admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.200501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.201117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.201495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.201942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.202657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.204371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.217177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1058/2101] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.222012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.226816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b99c960-df1e-4784-9ca2-c0bbdb0a19ed admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.234938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.235909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.236454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.236946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.237885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.241070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.241678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b076fd-60da-423a-b1db-61dd92994241 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.250397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74505e-1af8-4bb4-8670-fc6aa4ad7578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.251111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74505e-1af8-4bb4-8670-fc6aa4ad7578 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.251298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74505e-1af8-4bb4-8670-fc6aa4ad7578 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.251567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74505e-1af8-4bb4-8670-fc6aa4ad7578 admin admin] PATH_INFO: `/v3/projects/0c746d1e9ad64743b92c4f4f7845b5f0/users/0662f5c79107457393cbea8a22e20fa7/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.280487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1044/2102] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.282727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74505e-1af8-4bb4-8670-fc6aa4ad7578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c746d1e9ad64743b92c4f4f7845b5f0, user_id=0662f5c79107457393cbea8a22e20fa7, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.284971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.285541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74505e-1af8-4bb4-8670-fc6aa4ad7578 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.297938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f6486-f5e0-45a7-a097-017adafde2a9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.301873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1059/2103] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:59 2022] PUT /identity/v3/projects/0c746d1e9ad64743b92c4f4f7845b5f0/users/0662f5c79107457393cbea8a22e20fa7/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:59.314333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.314621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.314894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.345655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] MFA Rules not processed for user `0662f5c79107457393cbea8a22e20fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:59.346888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.353444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:59.403980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.405075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.405075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.405236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.405803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.407758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.421365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:59.434900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1045/2104] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.436585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.454059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1060/2105] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:46:59.454914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.456204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.458572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f5cd05-2a56-4d83-9992-5f3724eabc0f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.464253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.465002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.465323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.466185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.466841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.468760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.472593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680bc681-0fb9-45a7-8d53-9962586c747c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.483599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.484844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.485071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.485343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] PATH_INFO: `/v3/projects/ad1f06dea5c24d24addbd736669011d5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.486553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1f06dea5c24d24addbd736669011d5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.488821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.494040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1046/2106] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.494793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.513645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d04532-b8b1-4432-9700-04062b6757a7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.525831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.526427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.526714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.527129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.528654 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.530767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.551975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1f06dea5c24d24addbd736669011d5', 'request_id': 'req-219417db-06c3-4b53-ac0b-22ec212d078e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.565775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1f06dea5c24d24addbd736669011d5', 'request_id': 'req-219417db-06c3-4b53-ac0b-22ec212d078e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.567638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1f06dea5c24d24addbd736669011d5', 'request_id': 'req-219417db-06c3-4b53-ac0b-22ec212d078e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.569007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1047/2107] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.569851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.579853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] The token cache is being invalidate because project ad1f06dea5c24d24addbd736669011d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:46:59.580148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.580810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.582350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1061/2108] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:46:59 2022] DELETE /identity/v3/projects/ad1f06dea5c24d24addbd736669011d5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:59.583083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.584377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.638948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb34108-513b-4a07-af71-c7b62699e7e4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.644099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.644864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.645148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.645427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] PATH_INFO: `/v3/projects/88d701a36d7e47919bd4a335e415913f/users/f4630ed23bac4809a9b5fdb0d04ce9b6/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.654597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219417db-06c3-4b53-ac0b-22ec212d078e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.659734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.660155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.660443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.660715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.661352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.662622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=88d701a36d7e47919bd4a335e415913f, user_id=f4630ed23bac4809a9b5fdb0d04ce9b6, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.663496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.665367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.683223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1048/2109] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:59 2022] PUT /identity/v3/projects/88d701a36d7e47919bd4a335e415913f/users/f4630ed23bac4809a9b5fdb0d04ce9b6/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:59.684848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.686526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.689897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1062/2110] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:46:59.690766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.701777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b035e-9d54-4d87-8b37-ecaa1d33315a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.760914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:59.768321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9ab54-1f6c-41b2-b59e-e048e2b3b94e admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:46:59.772555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9532-e3f6-4e03-a772-c45af758ca89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.773174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9532-e3f6-4e03-a772-c45af758ca89 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.773491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9532-e3f6-4e03-a772-c45af758ca89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.773873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9532-e3f6-4e03-a772-c45af758ca89 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.774538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9532-e3f6-4e03-a772-c45af758ca89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.776600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:46:59.776924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9532-e3f6-4e03-a772-c45af758ca89 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.777250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.777528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.779056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] PATH_INFO: `/v3/users/ff77b9d2509e4ffaa8e830c7d75e364d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.781383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff77b9d2509e4ffaa8e830c7d75e364d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:46:59.785961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:46:59.792290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1063/2111] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:46:59.793914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:46:59.794700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:46:59.794700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:46:59.816257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:46:59.837427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] MFA Rules not processed for user `f4630ed23bac4809a9b5fdb0d04ce9b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:46:59.839806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.859448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff77b9d2509e4ffaa8e830c7d75e364d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.860154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff77b9d2509e4ffaa8e830c7d75e364d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.870168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff77b9d2509e4ffaa8e830c7d75e364d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.872207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff77b9d2509e4ffaa8e830c7d75e364d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.882579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff77b9d2509e4ffaa8e830c7d75e364d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:46:59.902928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1049/2112] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:46:59 2022] DELETE /identity/v3/users/ff77b9d2509e4ffaa8e830c7d75e364d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:46:59.915041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.918284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:46:59.919932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1064/2113] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:46:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:46:59.937231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:46:59.939128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.017357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac79620-9d6e-486a-875c-419cd2cda178 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.020659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0f0d9-5179-4584-8708-d527f3a719fb admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.028054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.029020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.029371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.029838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] PATH_INFO: `/v3/projects/74e14d756ce94b34b7e257e85a6844ad/users/0bb841492e3441218894de488a19d762/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.089147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e14d756ce94b34b7e257e85a6844ad, user_id=0bb841492e3441218894de488a19d762, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.099132 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.104507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.104980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.105261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.105530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.158473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1050/2114] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:46:59 2022] PUT /identity/v3/projects/74e14d756ce94b34b7e257e85a6844ad/users/0bb841492e3441218894de488a19d762/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:00.162428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.183957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:00.204515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.206014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.217877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1065/2115] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:46:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:00.269971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.272309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.296602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53aadc-1814-4202-816b-5d9896651c59 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.306875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead06e64-28aa-415a-8120-3ce7e2a6ab5d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.311441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.312334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.312556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.312969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.313252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.313956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.314367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.314749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.315595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] PATH_INFO: `/v3/users/f4630ed23bac4809a9b5fdb0d04ce9b6` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.315674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.316874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4630ed23bac4809a9b5fdb0d04ce9b6)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.319123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.330458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1051/2116] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:00.334892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.337389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.356645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61661cfa-82b4-4aac-9289-744796823ce0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.362561 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.363199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.363484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.363764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] PATH_INFO: `/v3/projects/681e577932364d41b73ea7dc684f8024` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.364969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=681e577932364d41b73ea7dc684f8024)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.366923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.377804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4630ed23bac4809a9b5fdb0d04ce9b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.378562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4630ed23bac4809a9b5fdb0d04ce9b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.379756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:00.388159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4630ed23bac4809a9b5fdb0d04ce9b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.389348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4630ed23bac4809a9b5fdb0d04ce9b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.406537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4630ed23bac4809a9b5fdb0d04ce9b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.421388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1066/2117] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:00 2022] DELETE /identity/v3/users/f4630ed23bac4809a9b5fdb0d04ce9b6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:00.422168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.423610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.445280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '681e577932364d41b73ea7dc684f8024', 'request_id': 'req-e9bf6384-45ac-4c7d-af0f-a706997d5219'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.454187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '681e577932364d41b73ea7dc684f8024', 'request_id': 'req-e9bf6384-45ac-4c7d-af0f-a706997d5219'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.462326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '681e577932364d41b73ea7dc684f8024', 'request_id': 'req-e9bf6384-45ac-4c7d-af0f-a706997d5219'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.485900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] The token cache is being invalidate because project 681e577932364d41b73ea7dc684f8024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:00.486168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.486950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e9bf6384-45ac-4c7d-af0f-a706997d5219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:00.489062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1052/2118] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:00 2022] DELETE /identity/v3/projects/681e577932364d41b73ea7dc684f8024 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:00.493342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48592374-15bf-4301-af6d-d1ef0c08e8c2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.493724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48592374-15bf-4301-af6d-d1ef0c08e8c2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.494002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48592374-15bf-4301-af6d-d1ef0c08e8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.516602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-48592374-15bf-4301-af6d-d1ef0c08e8c2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:00.517855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-48592374-15bf-4301-af6d-d1ef0c08e8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.524465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbbdc5e-5e4c-4776-acd1-a398c8aab121 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.526847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1053/2119] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:00.528642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a089a-ce69-4a11-986b-53d191b91e91 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.528996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a089a-ce69-4a11-986b-53d191b91e91 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.529371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44a089a-ce69-4a11-986b-53d191b91e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.530567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.531331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.531582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.531869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] PATH_INFO: `/v3/projects/74e14d756ce94b34b7e257e85a6844ad/users/0bb841492e3441218894de488a19d762/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.534085 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74e14d756ce94b34b7e257e85a6844ad, user_id=0bb841492e3441218894de488a19d762, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.538826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.559712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1067/2120] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:00 2022] PUT /identity/v3/projects/74e14d756ce94b34b7e257e85a6844ad/users/0bb841492e3441218894de488a19d762/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:00.560555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-c44a089a-ce69-4a11-986b-53d191b91e91 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:00.561414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.562024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c44a089a-ce69-4a11-986b-53d191b91e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.564231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.632125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1054/2121] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:00.633547 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.633859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.634130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.638583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec89f0-8586-43e3-ba69-283cb8f7fc6c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.642879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefe9589-96ad-4bc3-903b-3c41737169d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.643498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe9589-96ad-4bc3-903b-3c41737169d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.643783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe9589-96ad-4bc3-903b-3c41737169d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.644060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe9589-96ad-4bc3-903b-3c41737169d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.655206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe9589-96ad-4bc3-903b-3c41737169d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.656890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefe9589-96ad-4bc3-903b-3c41737169d6 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.661400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:00.669660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1068/2122] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:00.670173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] MFA Rules not processed for user `0bb841492e3441218894de488a19d762`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:00.671288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.671726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.671963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.672325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.694490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:00.695706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.703994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1069/2123] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:00.708064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.717751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8530e-bdcf-4fcb-91e2-e8e305b94ac3 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.722965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.723818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.723818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.724099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.724749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:00.725061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.726818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.741962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1070/2124] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:47:00.746176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.757728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e4cf64-1e3f-4a50-b9c7-a90fcdeed0a8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.763863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.764081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.764470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.764798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.765262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.766981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1055/2125] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:00.767299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.777799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.779780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.808879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1071/2126] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:00.810078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.812608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:00.861301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76450d8-2c3c-4909-9e81-85df486533c8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.888479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee1223-a0b8-419b-bbac-2f8f38dacefe admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.893708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.894330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.894612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.894873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.895546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.897727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.924940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1072/2127] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:00 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:00.928833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.931236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.932372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.932658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.932933 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.935186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:00.939669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cf5b3-24c2-4341-97d0-276910749dbb admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.946985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.948094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.948505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.948814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.949474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.951752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:00.962910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1073/2128] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:00.967570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:00.979988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d688ebf4-6209-4df8-82c3-7a6d6283ede8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:00.986223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:00.986830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:00.987110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:00.987383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:00.988030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:00.989650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.004067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.005978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.008980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1074/2129] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:01.013694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.020635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1056/2130] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:01.028908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0918fb-5be7-4e4a-844b-16959dea9dd8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.033895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b3252-bd09-4490-854b-13cbf057ff18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.034565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b3252-bd09-4490-854b-13cbf057ff18 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.034858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b3252-bd09-4490-854b-13cbf057ff18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.035117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b3252-bd09-4490-854b-13cbf057ff18 admin admin] PATH_INFO: `/v3/projects/53073d25cbc648a79e7c1503b9fde431/users/c4498124407448eeb38b8a28d747d19a/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.047965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.050303 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.051709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b3252-bd09-4490-854b-13cbf057ff18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53073d25cbc648a79e7c1503b9fde431, user_id=c4498124407448eeb38b8a28d747d19a, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.054795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b3252-bd09-4490-854b-13cbf057ff18 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.069523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f11130-327e-4598-bf0f-8e2e81b93095 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.082414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1075/2131] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:01 2022] PUT /identity/v3/projects/53073d25cbc648a79e7c1503b9fde431/users/c4498124407448eeb38b8a28d747d19a/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:01.083812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.084132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.084438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.123191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] MFA Rules not processed for user `c4498124407448eeb38b8a28d747d19a`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:01.124525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.149121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.149893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.150396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.150677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] PATH_INFO: `/v3/projects/88d701a36d7e47919bd4a335e415913f` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.152510 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88d701a36d7e47919bd4a335e415913f)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.155607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.189173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:01.205628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:01.220877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1076/2132] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:01.225428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.226195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88d701a36d7e47919bd4a335e415913f', 'request_id': 'req-eb19db58-4ce5-425c-853a-1f7572718f00'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:01.232637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88d701a36d7e47919bd4a335e415913f', 'request_id': 'req-eb19db58-4ce5-425c-853a-1f7572718f00'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:01.237045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68f1da-9bcb-4524-a48a-b81729d478b2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.240370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88d701a36d7e47919bd4a335e415913f', 'request_id': 'req-eb19db58-4ce5-425c-853a-1f7572718f00'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:01.256955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] The token cache is being invalidate because project 88d701a36d7e47919bd4a335e415913f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:01.257249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:01.258047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-eb19db58-4ce5-425c-853a-1f7572718f00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:01.260505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1057/2133] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:01 2022] DELETE /identity/v3/projects/88d701a36d7e47919bd4a335e415913f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:01.271394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab376b-de83-4408-afee-676abc499125 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.271394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab376b-de83-4408-afee-676abc499125 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.271394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab376b-de83-4408-afee-676abc499125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.289450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ecab376b-de83-4408-afee-676abc499125 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:01.290805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ecab376b-de83-4408-afee-676abc499125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.316255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.317676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.318106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.318522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.319409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.322059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.355848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1077/2134] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/projects => generated 387 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:01.361244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.363171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.368104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1058/2135] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:01.377380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271646-083d-4bee-b314-38a00285f47d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.377702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271646-083d-4bee-b314-38a00285f47d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.377979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83271646-083d-4bee-b314-38a00285f47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.411785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-83271646-083d-4bee-b314-38a00285f47d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:01.417972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-83271646-083d-4bee-b314-38a00285f47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.456629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76b8b5-a825-4071-aa80-3b81bb72ce98 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.462384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.463028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.463356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.463631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.464294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.466324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.493698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1078/2136] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:01.495209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.504096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1059/2137] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:01.505131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271646-083d-4bee-b314-38a00285f47d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.510450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc581413-8868-40d4-b4ec-379a342f7d12 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.515036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83271646-083d-4bee-b314-38a00285f47d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.516479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.517163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.517447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.517754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.518401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.519884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.520050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.520575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.520857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.521131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.521864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:01.522108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.523741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.531727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1079/2138] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:01.535575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.536298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1060/2139] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:47:01.540630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.546820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a125ad-a01c-46b1-b3a0-5de480ef1ec5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.551606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec74978-3afa-4f60-9670-fcd5e2ec8fdc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.552378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.553012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.553294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.553588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.554229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.555844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.556402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.556996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.557289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.557546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.558185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.560658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.568101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1080/2140] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:01.574076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.585884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540f246-f862-4513-9dfd-0515076721b7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.592053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045370d-479b-4395-96d9-99709ac0bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.592729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045370d-479b-4395-96d9-99709ac0bd50 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.593008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045370d-479b-4395-96d9-99709ac0bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.593286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045370d-479b-4395-96d9-99709ac0bd50 admin admin] PATH_INFO: `/v3/projects/ac4b8ee0363c4bd5abfcae608507f8ae/users/3efcdcdcda2a424f877ba5ea05bc25c4/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.604526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1061/2141] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:01.605608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.613749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045370d-479b-4395-96d9-99709ac0bd50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4b8ee0363c4bd5abfcae608507f8ae, user_id=3efcdcdcda2a424f877ba5ea05bc25c4, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.616705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045370d-479b-4395-96d9-99709ac0bd50 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.636164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:01.638661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1081/2142] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:01 2022] PUT /identity/v3/projects/ac4b8ee0363c4bd5abfcae608507f8ae/users/3efcdcdcda2a424f877ba5ea05bc25c4/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:01.650412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.650781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.651155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.682382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f21b92-84de-4a4e-accd-90baec34e1a6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.683385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] MFA Rules not processed for user `3efcdcdcda2a424f877ba5ea05bc25c4`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:01.685457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:01.693046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:01.752289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.752907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.753195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.753465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.755041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.758401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.768321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1082/2143] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:01.771599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.783359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f21a5-e731-4c6a-9151-d4d765cc2917 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.786007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1062/2144] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/users => generated 288 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:01.789623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.790283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.790669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.790954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.791818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.792469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.795376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.801708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e27f0e-7acf-4451-849d-66fb13befbbe admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.807651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.808235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.808517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.808791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.811150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.813896 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.825785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1063/2145] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:01.828647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.836633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1083/2146] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:01.839578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c807a-ce35-4a60-8b5f-848420393e9c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.842840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.919545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.920163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.920443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.920716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.921506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.923006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.927239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf77108-0684-47df-9884-23938f323bd4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.932121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.933192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.933192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.933367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.937433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.937595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1064/2147] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:01.938035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.938485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.950031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f714ed-9f76-4861-be68-d0cbeb3d35b7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.956213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.957475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.958072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.958438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] PATH_INFO: `/v3/projects/873d8330010249f58ac18c8e17946f6e/users/2c9fa86b65ec431eb3f77cf2bfe5cda4/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.961892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1084/2148] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:01 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:01.965664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:01.977378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62006906-cc4d-4a37-9087-405aec726b2b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:01.977965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] RBAC: Authorizing `identity:create_grant(project_id=873d8330010249f58ac18c8e17946f6e, user_id=2c9fa86b65ec431eb3f77cf2bfe5cda4, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.981456 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:01.984257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2253bd-4f5d-454f-adfb-66cfe556d265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:01.985095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2253bd-4f5d-454f-adfb-66cfe556d265 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:01.985441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2253bd-4f5d-454f-adfb-66cfe556d265 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:01.985809 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2253bd-4f5d-454f-adfb-66cfe556d265 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:01.986705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2253bd-4f5d-454f-adfb-66cfe556d265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:01.988869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2253bd-4f5d-454f-adfb-66cfe556d265 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.001661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1065/2149] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:01 2022] PUT /identity/v3/projects/873d8330010249f58ac18c8e17946f6e/users/2c9fa86b65ec431eb3f77cf2bfe5cda4/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:02.005596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1085/2150] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:02.008313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.039816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:02.073027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.073484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.073622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.098400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bc80a6-6819-4098-ae76-5b128bef792a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.104315 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.104987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.105273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.105714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] PATH_INFO: `/v3/projects/fe579d9ac70c4a7b924950d37d5ba91b/users/47ea6173b8474eaab5a955cdd4d642b3/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.116866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] MFA Rules not processed for user `2c9fa86b65ec431eb3f77cf2bfe5cda4`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:02.118149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:02.125607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe579d9ac70c4a7b924950d37d5ba91b, user_id=47ea6173b8474eaab5a955cdd4d642b3, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.128703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.142858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1066/2151] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:02 2022] PUT /identity/v3/projects/fe579d9ac70c4a7b924950d37d5ba91b/users/47ea6173b8474eaab5a955cdd4d642b3/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:02.146695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.184520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:02.197881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1086/2152] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:02.208394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.210421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:02.223237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad01c8-e650-42e3-937e-1638587f23af admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.228692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.229640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.229942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.230215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.232005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.234424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.245455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1067/2153] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:02.249356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.269867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f64c83-4cec-4fa5-abbd-f1008993a8b5 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.275273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912ff1f7-2969-4cf5-97cd-c7df0f4857a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.275967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ff1f7-2969-4cf5-97cd-c7df0f4857a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.276251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ff1f7-2969-4cf5-97cd-c7df0f4857a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.276528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912ff1f7-2969-4cf5-97cd-c7df0f4857a9 admin admin] PATH_INFO: `/v3/projects/fe579d9ac70c4a7b924950d37d5ba91b/users/47ea6173b8474eaab5a955cdd4d642b3/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.279022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ff1f7-2969-4cf5-97cd-c7df0f4857a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe579d9ac70c4a7b924950d37d5ba91b, user_id=47ea6173b8474eaab5a955cdd4d642b3, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.281704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912ff1f7-2969-4cf5-97cd-c7df0f4857a9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.286542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46150eb9-ed9e-4557-81ec-18d49093d605 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.293103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1068/2154] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:02 2022] PUT /identity/v3/projects/fe579d9ac70c4a7b924950d37d5ba91b/users/47ea6173b8474eaab5a955cdd4d642b3/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:02.304624 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.304911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.305189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.334050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] MFA Rules not processed for user `47ea6173b8474eaab5a955cdd4d642b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:02.335261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:02.336625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.337234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.337514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.337814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.395752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.397717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.406053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1069/2155] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:02.407288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1087/2156] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:02.417404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:02.419441 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:02.432061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724388a-af34-4c44-819d-d381f881bda7 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:02.435821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0782e-770a-44f7-9cb5-19f65b382b30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:02.436442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0782e-770a-44f7-9cb5-19f65b382b30 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:02.436709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0782e-770a-44f7-9cb5-19f65b382b30 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:02.436983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0782e-770a-44f7-9cb5-19f65b382b30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:02.446574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0782e-770a-44f7-9cb5-19f65b382b30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:02.448012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0782e-770a-44f7-9cb5-19f65b382b30 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:02.454789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:02.463096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1070/2157] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:03.159700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:03.169058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010f4ea-6397-4bab-93d3-8db200419b67 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:03.172784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:03.173396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.173719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.173995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.175784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:03.239928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:03.241430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:03.251742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1088/2158] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:03 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:03.833706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:03.834151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:03.834241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:03.853524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] MFA Rules not processed for user `2c9fa86b65ec431eb3f77cf2bfe5cda4`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:03.854815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:03.862070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1071/2159] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:05.695519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:05.699875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:05.705631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36ff2a-f2df-4a7e-a479-cfef2f37856d service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:05.709279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:05.710032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:05.710316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:05.710590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:05.712461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:05.778765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:05.780206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:05.790171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1089/2160] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:08.417412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:08.427690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12381eb0-ab7c-44fd-aa54-c1f4560166a9 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:08.432067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:08.432863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:08.433144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:08.433418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:08.498605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:08.500059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:08.509304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1072/2161] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:08 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:10.708679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:10.710664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:10.722645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:10.805693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbfd97d-a421-4a26-bd62-a5a5723bbb9a service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:10.811016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:10.811658 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.812075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.812391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] PATH_INFO: `/v3/users/0662f5c79107457393cbea8a22e20fa7` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.813497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0662f5c79107457393cbea8a22e20fa7)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:10.815467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:10.815898 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb5907-5cc4-48de-8955-bb00907c818e service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:10.824281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:10.825152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.825259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.825519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] PATH_INFO: `/v3/users/c4498124407448eeb38b8a28d747d19a` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.827033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4498124407448eeb38b8a28d747d19a)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:10.829542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:10.882037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0662f5c79107457393cbea8a22e20fa7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.883248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0662f5c79107457393cbea8a22e20fa7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.906691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4498124407448eeb38b8a28d747d19a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.907428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4498124407448eeb38b8a28d747d19a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.907849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0662f5c79107457393cbea8a22e20fa7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.909674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0662f5c79107457393cbea8a22e20fa7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.917807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4498124407448eeb38b8a28d747d19a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.919237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4498124407448eeb38b8a28d747d19a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.935258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-349773d6-6fbb-4a53-98db-0921a762f2c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4498124407448eeb38b8a28d747d19a'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.935496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-07eb6b12-fb60-4bfe-bbe9-a6a8e4d45b00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0662f5c79107457393cbea8a22e20fa7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:10.956935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1090/2162] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:10 2022] DELETE /identity/v3/users/0662f5c79107457393cbea8a22e20fa7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:10.961681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.962052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.962391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.976986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1073/2163] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:10 2022] DELETE /identity/v3/users/c4498124407448eeb38b8a28d747d19a => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:10.979544 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:10.979900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:10.980645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:10.986966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:10.988283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:10.999040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:11.003534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:11.005115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:11.011266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:11.067774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1091/2164] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:11.075812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.077757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:11.093533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1074/2165] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:11.142094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.143777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:11.162188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea9405f-3e77-4de6-bc4e-4e330398ae9c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.166322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.166943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.167252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.168905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.181339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.184549 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.198772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1092/2166] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:11.224503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b70839-02eb-4fc8-a7a0-8c300a501ed5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.229425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.230336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.230619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.231581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.243820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.245312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.257952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1075/2167] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:11.769513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.771119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:11.800213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522c789-cd75-4492-8b47-7fc6194db4bc service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.808400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.810117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.810529 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.810869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] PATH_INFO: `/v3/projects/0c746d1e9ad64743b92c4f4f7845b5f0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.812486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c746d1e9ad64743b92c4f4f7845b5f0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.814745 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:11.868205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c746d1e9ad64743b92c4f4f7845b5f0', 'request_id': 'req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:11.872510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c746d1e9ad64743b92c4f4f7845b5f0', 'request_id': 'req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:11.880179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c746d1e9ad64743b92c4f4f7845b5f0', 'request_id': 'req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:11.892026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] The token cache is being invalidate because project 0c746d1e9ad64743b92c4f4f7845b5f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:11.892311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:11.892948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:11.894448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1093/2168] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:11 2022] DELETE /identity/v3/projects/0c746d1e9ad64743b92c4f4f7845b5f0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:11.898608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:11.910213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d0405-6bc0-4515-b9bf-f96bc825fdeb service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:11.975316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:11.975648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:11.975927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:11.976202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] PATH_INFO: `/v3/users/0bb841492e3441218894de488a19d762` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:11.977342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bb841492e3441218894de488a19d762)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:11.979307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:12.035803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bb841492e3441218894de488a19d762'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.036499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bb841492e3441218894de488a19d762'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.044444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bb841492e3441218894de488a19d762'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.045131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bb841492e3441218894de488a19d762'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.046859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:12.054477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bb841492e3441218894de488a19d762'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.074034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1076/2169] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:11 2022] DELETE /identity/v3/users/0bb841492e3441218894de488a19d762 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:12.380117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:12.381632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:12.398948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ee8e1-6f3d-41d8-81cb-c9b83d3105d2 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:12.471022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:12.492385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:12.492943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.493220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.493489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] PATH_INFO: `/v3/projects/74e14d756ce94b34b7e257e85a6844ad` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.494677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] RBAC: Authorizing `identity:delete_project(project_id=74e14d756ce94b34b7e257e85a6844ad)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:12.497230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:12.547046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:12.548553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:12.566354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2636ab-ce66-47bd-b70a-41f3a1f2d16b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:12.575465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74e14d756ce94b34b7e257e85a6844ad', 'request_id': 'req-8dc55710-226c-4566-8b1c-1322b146202c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.592724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74e14d756ce94b34b7e257e85a6844ad', 'request_id': 'req-8dc55710-226c-4566-8b1c-1322b146202c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.618652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74e14d756ce94b34b7e257e85a6844ad', 'request_id': 'req-8dc55710-226c-4566-8b1c-1322b146202c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.649213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] The token cache is being invalidate because project 74e14d756ce94b34b7e257e85a6844ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:12.650213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.650978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8dc55710-226c-4566-8b1c-1322b146202c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.658515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1094/2170] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:12 2022] DELETE /identity/v3/projects/74e14d756ce94b34b7e257e85a6844ad => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:12.660010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072202e-21ef-499c-bde5-2506aec3faa4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.660208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072202e-21ef-499c-bde5-2506aec3faa4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.660491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b072202e-21ef-499c-bde5-2506aec3faa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.697838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:12.698519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.698804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.700450 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] PATH_INFO: `/v3/projects/53073d25cbc648a79e7c1503b9fde431` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.703390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53073d25cbc648a79e7c1503b9fde431)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:12.704027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b072202e-21ef-499c-bde5-2506aec3faa4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:12.705351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:12.705897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b072202e-21ef-499c-bde5-2506aec3faa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:12.716637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1095/2171] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:12.725144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.725424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.725727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.752120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:12.752120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:12.769656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53073d25cbc648a79e7c1503b9fde431', 'request_id': 'req-0d0566f2-6dd8-4705-bd0d-574b231dece4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.777704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53073d25cbc648a79e7c1503b9fde431', 'request_id': 'req-0d0566f2-6dd8-4705-bd0d-574b231dece4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.787996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53073d25cbc648a79e7c1503b9fde431', 'request_id': 'req-0d0566f2-6dd8-4705-bd0d-574b231dece4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.803664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] The token cache is being invalidate because project 53073d25cbc648a79e7c1503b9fde431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:12.804057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.805110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:12.810876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1077/2172] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:12 2022] DELETE /identity/v3/projects/53073d25cbc648a79e7c1503b9fde431 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:12.811992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:12.813419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:12.842203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1096/2173] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:12.846339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:12.861240 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d6382-2568-40a6-bc33-2651c63cf6cc None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:12.868845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:12.870238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.872537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.872811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.873477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:12.873802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:12.876459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:12.908766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1097/2174] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:47:12.913536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:12.924927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0566f2-6dd8-4705-bd0d-574b231dece4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:12.930195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:12.931832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.932310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.932773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] PATH_INFO: `/v3/users/3efcdcdcda2a424f877ba5ea05bc25c4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.934912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3efcdcdcda2a424f877ba5ea05bc25c4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:12.937039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:12.940619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790c3c3-ff48-47a3-926b-00ee6ba3ea18 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:12.942868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:12.953103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:12.953897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:12.954315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:12.954937 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:12.955903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:12.968675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.042032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3efcdcdcda2a424f877ba5ea05bc25c4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.043682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3efcdcdcda2a424f877ba5ea05bc25c4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.045609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1098/2175] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:12 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:13.049433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.052056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:13.078228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3efcdcdcda2a424f877ba5ea05bc25c4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.080143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3efcdcdcda2a424f877ba5ea05bc25c4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.094261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3efcdcdcda2a424f877ba5ea05bc25c4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.108117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:13.154558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1078/2176] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:12 2022] DELETE /identity/v3/users/3efcdcdcda2a424f877ba5ea05bc25c4 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:13.180593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d28238-5689-45cf-a3ff-7f13d0d03c33 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.274882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.275535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.275967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.276269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.276910 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.279475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.308680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1099/2177] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:13 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:13.313442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.341546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccc3414-9e81-4e91-99cb-c113a93362b7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.346781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.347560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.347841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.348113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.348913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.350658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.365497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1079/2178] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:13.366986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.378835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeac0c8-8f9c-4c52-9dc3-fe7c178cf07e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.385853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f5b9-0ac8-4812-86fb-b71e818dd101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.387498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f5b9-0ac8-4812-86fb-b71e818dd101 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.387498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f5b9-0ac8-4812-86fb-b71e818dd101 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.387961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f5b9-0ac8-4812-86fb-b71e818dd101 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.388782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f5b9-0ac8-4812-86fb-b71e818dd101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.390616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f5b9-0ac8-4812-86fb-b71e818dd101 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.405373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1100/2179] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:13.406346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.418191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef6e8d-09fb-4ce1-9791-d852c69f4a33 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.424123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.425041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.425490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.426059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] PATH_INFO: `/v3/projects/3b32dfdcb49b42f69eaf7fea74140946/users/c5cb880f5fad4ceba32633c21c9b5b86/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.447195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b32dfdcb49b42f69eaf7fea74140946, user_id=c5cb880f5fad4ceba32633c21c9b5b86, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.451021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.475296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1080/2180] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:13 2022] PUT /identity/v3/projects/3b32dfdcb49b42f69eaf7fea74140946/users/c5cb880f5fad4ceba32633c21c9b5b86/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:13.487609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.487904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.488187 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.515374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.517299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:13.526092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] MFA Rules not processed for user `c5cb880f5fad4ceba32633c21c9b5b86`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:13.527699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:13.618099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769472a-b665-4c27-93d3-e756fc5fbd59 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.621079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1101/2181] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:13.623678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.624445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.624857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.625168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] PATH_INFO: `/v3/projects/ac4b8ee0363c4bd5abfcae608507f8ae` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.626338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4b8ee0363c4bd5abfcae608507f8ae)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.628256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.633998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.635632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:13.685173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4b8ee0363c4bd5abfcae608507f8ae', 'request_id': 'req-90615388-1fe3-4ebd-92eb-e687f5187562'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.686493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:13.692351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4b8ee0363c4bd5abfcae608507f8ae', 'request_id': 'req-90615388-1fe3-4ebd-92eb-e687f5187562'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.696497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:13.701039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9a2d4-50bb-48a9-b77a-c7c82a6e17ad None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.703614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4b8ee0363c4bd5abfcae608507f8ae', 'request_id': 'req-90615388-1fe3-4ebd-92eb-e687f5187562'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.720354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] The token cache is being invalidate because project ac4b8ee0363c4bd5abfcae608507f8ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:13.720724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.721542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.723141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1081/2182] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:13 2022] DELETE /identity/v3/projects/ac4b8ee0363c4bd5abfcae608507f8ae => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:13.726906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:13.729801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:13.778328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.779352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.779590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.779938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.781881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:13.813414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90615388-1fe3-4ebd-92eb-e687f5187562 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:13.818486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:13.819338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:13.819728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:13.820138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] PATH_INFO: `/v3/users/47ea6173b8474eaab5a955cdd4d642b3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:13.821398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=47ea6173b8474eaab5a955cdd4d642b3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.823622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.858900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:13.860225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:13.876568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1102/2183] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:13.902217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47ea6173b8474eaab5a955cdd4d642b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.904412 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47ea6173b8474eaab5a955cdd4d642b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.913232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47ea6173b8474eaab5a955cdd4d642b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.914312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47ea6173b8474eaab5a955cdd4d642b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.923621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8ea847aa-8875-4033-b7b2-57f1b942ebbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47ea6173b8474eaab5a955cdd4d642b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:13.945765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1082/2184] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:13 2022] DELETE /identity/v3/users/47ea6173b8474eaab5a955cdd4d642b3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:14.414179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.428373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2ddf8-4e60-4328-bd55-7df9ab2b5a9e service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.487470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.488109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.488392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.488668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] PATH_INFO: `/v3/projects/fe579d9ac70c4a7b924950d37d5ba91b` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.489817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe579d9ac70c4a7b924950d37d5ba91b)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.491729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:14.537478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe579d9ac70c4a7b924950d37d5ba91b', 'request_id': 'req-de16974b-6638-4ee6-870a-6c84db1f4036'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:14.542882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe579d9ac70c4a7b924950d37d5ba91b', 'request_id': 'req-de16974b-6638-4ee6-870a-6c84db1f4036'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:14.545113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:14.551988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe579d9ac70c4a7b924950d37d5ba91b', 'request_id': 'req-de16974b-6638-4ee6-870a-6c84db1f4036'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:14.564096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] The token cache is being invalidate because project fe579d9ac70c4a7b924950d37d5ba91b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:14.564390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:14.565023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-de16974b-6638-4ee6-870a-6c84db1f4036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:14.566545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1103/2185] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:14 2022] DELETE /identity/v3/projects/fe579d9ac70c4a7b924950d37d5ba91b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:14.572889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.573182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.573449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.592660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:14.593895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:14.646483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:14.659029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1083/2186] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:14.667410 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.667695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.667970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.688175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:14.689365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:14.697426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1104/2187] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:14.701588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.711047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ace2dd-291e-4008-8219-df6f259676b9 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.715593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.716181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.716455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.716720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.717354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:14.717678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.719259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:14.730500 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1084/2188] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:47:14.735558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.745441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fada3e-aa01-498d-bf0b-08fb5b39268c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.751226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.751832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.752112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.752382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.753019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.755007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:14.783325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1105/2189] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:14 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:14.787686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.798034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e26b7fe-38dc-40c9-b5da-58caaa04bd53 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.857020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.857791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.858159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.858777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.859409 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.861327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:14.886880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1085/2190] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:14 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:14.891506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.902368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a5e04-afad-4845-b82f-a0ff06982c85 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.907526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.908141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.908425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.908698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.909337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.911056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:14.925114 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1106/2191] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:14.926327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.937088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ea90a3-ca2c-42d2-8550-9db51e1f0757 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.941724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca4edf-d3a2-4f20-abc3-19992e622f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.942443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4edf-d3a2-4f20-abc3-19992e622f48 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.942756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4edf-d3a2-4f20-abc3-19992e622f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.943059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca4edf-d3a2-4f20-abc3-19992e622f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.943700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca4edf-d3a2-4f20-abc3-19992e622f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.945305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca4edf-d3a2-4f20-abc3-19992e622f48 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:14.956795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1086/2192] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:14.960915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:14.972080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa980d-4120-4f02-ae44-98d347c48ac3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:14.977381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:14.978164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:14.978487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:14.978815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] PATH_INFO: `/v3/projects/33a4f0e0d56743d587f90e129055dc06/users/8c5da033360f4bba808bfdfdef2b0ba3/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:14.994520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] RBAC: Authorizing `identity:create_grant(project_id=33a4f0e0d56743d587f90e129055dc06, user_id=8c5da033360f4bba808bfdfdef2b0ba3, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:14.997794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:15.011508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1107/2193] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:14 2022] PUT /identity/v3/projects/33a4f0e0d56743d587f90e129055dc06/users/8c5da033360f4bba808bfdfdef2b0ba3/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:15.017034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06d66-c47d-44f9-95b7-837920986abc None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:15.017323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06d66-c47d-44f9-95b7-837920986abc None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:15.017622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b06d66-c47d-44f9-95b7-837920986abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:15.037829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-86b06d66-c47d-44f9-95b7-837920986abc None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:15.038951 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-86b06d66-c47d-44f9-95b7-837920986abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:15.102949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1087/2194] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:15.112682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:15.114454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:15.171146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebac3e8f-367a-47af-89c7-f9eb8c35510c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:15.174793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:15.175395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:15.175673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:15.175942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:15.185391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:15.186836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:15.196055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1108/2195] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:16.005305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.005868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.005985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.035079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] MFA Rules not processed for user `8c5da033360f4bba808bfdfdef2b0ba3`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:16.036312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:16.097423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:16.119063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1088/2196] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:16.123390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:16.133851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf5ed63-9342-42d6-9b43-a8031339efad service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:16.140373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:16.140985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.141263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.141837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.142484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:16.144379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:16.176228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1109/2197] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:16 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:16.179921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:16.192216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04f716-b495-4bf5-b5cf-a24c1c1eba0e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:16.254078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:16.254696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.254979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.255253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.255882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:16.258149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:16.283865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1089/2198] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:16 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:16.287464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:16.297768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b96e95f-3f81-4342-ab2a-ce63b6b8343a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:16.302316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:16.302916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.303314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.303805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.304492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:16.306122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:16.318278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1110/2199] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:16.322410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:16.333531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cb114-6978-4edb-9ed8-b98c63b316f0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:16.338540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:16.339199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.339481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.339757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] PATH_INFO: `/v3/projects/d34e44bd4f65471b9c8560eee7fd26dc/users/4f0a8e275cdb45a393b63cc4e1040a58/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.354733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34e44bd4f65471b9c8560eee7fd26dc, user_id=4f0a8e275cdb45a393b63cc4e1040a58, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:16.357453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:16.371740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1090/2200] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:16 2022] PUT /identity/v3/projects/d34e44bd4f65471b9c8560eee7fd26dc/users/4f0a8e275cdb45a393b63cc4e1040a58/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:16.375687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:16.385699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bed9ab-5b84-42da-819d-4903316c94a2 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:16.433107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:16.444034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601df3b-5d0b-44e7-8ab0-b8437bcd198b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:16.444643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601df3b-5d0b-44e7-8ab0-b8437bcd198b admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.444924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601df3b-5d0b-44e7-8ab0-b8437bcd198b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.445225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601df3b-5d0b-44e7-8ab0-b8437bcd198b admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.445894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601df3b-5d0b-44e7-8ab0-b8437bcd198b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:16.447485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c601df3b-5d0b-44e7-8ab0-b8437bcd198b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:16.457736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1111/2201] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:16.462832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:16.473085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fe209d-d4a0-432a-97f8-e247e7053ffe admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:16.477605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:16.478248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:16.478523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:16.478794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] PATH_INFO: `/v3/projects/d34e44bd4f65471b9c8560eee7fd26dc/users/4f0a8e275cdb45a393b63cc4e1040a58/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:16.480682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34e44bd4f65471b9c8560eee7fd26dc, user_id=4f0a8e275cdb45a393b63cc4e1040a58, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:16.483604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:16.493325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1091/2202] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:16 2022] PUT /identity/v3/projects/d34e44bd4f65471b9c8560eee7fd26dc/users/4f0a8e275cdb45a393b63cc4e1040a58/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:17.241476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:17.241963 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:17.242153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:17.273251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] MFA Rules not processed for user `4f0a8e275cdb45a393b63cc4e1040a58`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:17.274706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:17.359685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1112/2203] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:17.367197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:17.369207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:17.438471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324c60f6-3653-4cf5-8e87-2ca74f7fcad9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:17.442543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0fd2a6-0fb1-4679-9cfc-9e33bd28d822 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:17.443235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fd2a6-0fb1-4679-9cfc-9e33bd28d822 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:17.443527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fd2a6-0fb1-4679-9cfc-9e33bd28d822 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:17.443825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0fd2a6-0fb1-4679-9cfc-9e33bd28d822 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:17.506343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fd2a6-0fb1-4679-9cfc-9e33bd28d822 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:17.507738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0fd2a6-0fb1-4679-9cfc-9e33bd28d822 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:17.512791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:17.517467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1092/2204] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:19.951047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:19.964557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095e5e5-8d13-41a2-b616-55b98784752c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:19.969065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:19.969830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:19.970200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:19.970701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:19.982614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:19.985351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:19.997858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1113/2205] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:21.703233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae57bf8-e6fd-45a5-9312-f53dbd8b3aac None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:21.704053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae57bf8-e6fd-45a5-9312-f53dbd8b3aac None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:21.704513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae57bf8-e6fd-45a5-9312-f53dbd8b3aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:21.747982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5ae57bf8-e6fd-45a5-9312-f53dbd8b3aac None None] MFA Rules not processed for user `a621860d8fcb4327820ce6b802e6a725`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:21.754836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5ae57bf8-e6fd-45a5-9312-f53dbd8b3aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:21.864810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1093/2206] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:22.847664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:22.850265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:22.955676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538360fd-abda-4fb7-90a7-f8ee13aba918 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:22.960689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:22.961339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:22.961666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:22.962097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] PATH_INFO: `/v3/users/c5cb880f5fad4ceba32633c21c9b5b86` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:22.963396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cb880f5fad4ceba32633c21c9b5b86)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:22.965451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:23.040824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cb880f5fad4ceba32633c21c9b5b86'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.041808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cb880f5fad4ceba32633c21c9b5b86'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.051450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cb880f5fad4ceba32633c21c9b5b86'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.052124 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cb880f5fad4ceba32633c21c9b5b86'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.061585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:23.070683 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cb880f5fad4ceba32633c21c9b5b86'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.093306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1114/2207] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:22 2022] DELETE /identity/v3/users/c5cb880f5fad4ceba32633c21c9b5b86 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:23.098630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:23.099302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:23.099590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:23.120366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:23.121686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:23.190965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1094/2208] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:23.201643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:23.203082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:23.284712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a580d3-4b97-44a9-89a6-53173ec5ebfd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:23.288432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac6ea1-52ac-427d-aa1a-313e9aecadc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:23.289101 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6ea1-52ac-427d-aa1a-313e9aecadc2 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:23.289340 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6ea1-52ac-427d-aa1a-313e9aecadc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:23.289656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac6ea1-52ac-427d-aa1a-313e9aecadc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:23.304694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6ea1-52ac-427d-aa1a-313e9aecadc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:23.306812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac6ea1-52ac-427d-aa1a-313e9aecadc2 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:23.322446 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1115/2209] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:23.771194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:23.772925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:23.786944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a78791-934e-4c89-ba9f-f8669db8252b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:23.792736 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:23.794055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:23.794397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:23.794745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] PATH_INFO: `/v3/projects/3b32dfdcb49b42f69eaf7fea74140946` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:23.795964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b32dfdcb49b42f69eaf7fea74140946)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:23.798301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:23.864666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b32dfdcb49b42f69eaf7fea74140946', 'request_id': 'req-e23a6b33-b111-406b-8c49-8a4eda7441b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.871644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b32dfdcb49b42f69eaf7fea74140946', 'request_id': 'req-e23a6b33-b111-406b-8c49-8a4eda7441b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.878859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b32dfdcb49b42f69eaf7fea74140946', 'request_id': 'req-e23a6b33-b111-406b-8c49-8a4eda7441b3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.896489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] The token cache is being invalidate because project 3b32dfdcb49b42f69eaf7fea74140946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:23.896938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.897705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e23a6b33-b111-406b-8c49-8a4eda7441b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:23.902853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1095/2210] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:23 2022] DELETE /identity/v3/projects/3b32dfdcb49b42f69eaf7fea74140946 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:23.904927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:23.905414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:23.905508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:23.942708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:23.943946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:24.041515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1116/2211] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:24.049678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.049963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.050238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.071114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:24.072574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:24.087961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1096/2212] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:24.092635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.126141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b732077c-ac62-471c-80fb-c5c912dbac43 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.142505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.143134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.143581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.144007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.144882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:24.145494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.147386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.177909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1117/2213] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:47:24.181244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.198200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3873f-a18b-40b6-8f8b-10ef9a31cbcf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.210872 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.211796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.212245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.212669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.213735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.216971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.248298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:24.270694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1097/2214] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:24 2022] POST /identity/v3/projects => generated 369 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:24.272414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.286779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825c1bc-ab3d-4b9b-bc1a-c1d150b6e489 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.411545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.413275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.413545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.413870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.415147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.419643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.445799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:24.451943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1118/2215] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:24 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:24.455445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.468143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f03c9c-4d65-4994-b500-676d3b79b726 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.473891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.474991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.475390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.475786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.476680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.478421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.491978 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1098/2216] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:24.496612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.509264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1cccdc-db72-4686-8ab9-5396ec5e4287 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.526263 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98203543-983e-4b85-a07c-16629154d2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.527003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98203543-983e-4b85-a07c-16629154d2fd admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.530309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98203543-983e-4b85-a07c-16629154d2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.530309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98203543-983e-4b85-a07c-16629154d2fd admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.530309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98203543-983e-4b85-a07c-16629154d2fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.530652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98203543-983e-4b85-a07c-16629154d2fd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.553808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1119/2217] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:24.558169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.569534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b5599-0b3c-475c-a909-6d0584001868 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.577939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.578635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.578914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.579499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] PATH_INFO: `/v3/projects/58b80d12a1534241b8b9e7ebb9950062/users/2c7d0f3ad00e44c2827553d7f0e73ff3/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.599910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58b80d12a1534241b8b9e7ebb9950062, user_id=2c7d0f3ad00e44c2827553d7f0e73ff3, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.603449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.622843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1099/2218] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:24 2022] PUT /identity/v3/projects/58b80d12a1534241b8b9e7ebb9950062/users/2c7d0f3ad00e44c2827553d7f0e73ff3/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:24.634472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3927395-5810-47f2-8c51-7a0a736395b4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.634875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3927395-5810-47f2-8c51-7a0a736395b4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.635310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3927395-5810-47f2-8c51-7a0a736395b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.667185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f3927395-5810-47f2-8c51-7a0a736395b4 None None] MFA Rules not processed for user `2c7d0f3ad00e44c2827553d7f0e73ff3`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:24.668679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f3927395-5810-47f2-8c51-7a0a736395b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:24.752678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1120/2219] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:24.766073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:24.768331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:24.874477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7f79f-f4c5-456f-95c3-92bf9013d804 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:24.878169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b373d9fd-e93f-4987-a99c-54b8e2f69aa1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:24.878773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373d9fd-e93f-4987-a99c-54b8e2f69aa1 service glance] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:24.879051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373d9fd-e93f-4987-a99c-54b8e2f69aa1 service glance] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:24.879322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b373d9fd-e93f-4987-a99c-54b8e2f69aa1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:24.889220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373d9fd-e93f-4987-a99c-54b8e2f69aa1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:24.890907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b373d9fd-e93f-4987-a99c-54b8e2f69aa1 service glance] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:24.901326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1100/2220] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:26.612269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f583e74-d633-465d-8cd6-4a7c62b10d94 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:26.612828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f583e74-d633-465d-8cd6-4a7c62b10d94 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:26.612828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f583e74-d633-465d-8cd6-4a7c62b10d94 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:26.614380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1121/2221] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:47:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 05:47:26.625442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166d914-c52d-42de-bf77-37da037f385d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:26.625990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166d914-c52d-42de-bf77-37da037f385d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:26.626374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166d914-c52d-42de-bf77-37da037f385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:26.662345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d166d914-c52d-42de-bf77-37da037f385d None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:26.663867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d166d914-c52d-42de-bf77-37da037f385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:26.706429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f583e74-d633-465d-8cd6-4a7c62b10d94 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:26.708579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1f583e74-d633-465d-8cd6-4a7c62b10d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:26.784201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1101/2222] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:47:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:26.798254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166d914-c52d-42de-bf77-37da037f385d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:26.800076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d166d914-c52d-42de-bf77-37da037f385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:26.815170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166d914-c52d-42de-bf77-37da037f385d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:26.830193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:26.830865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:26.831213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:26.831503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:26.842127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:26.844170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:26.855157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1102/2223] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:26.872403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f583e74-d633-465d-8cd6-4a7c62b10d94 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:26.880997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:26.882278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:26.882714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:26.883136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] PATH_INFO: `/v3/users/914bea2efd334e6c8dc31698dfabbf8e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:26.901011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=914bea2efd334e6c8dc31698dfabbf8e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:26.901011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:26.954112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914bea2efd334e6c8dc31698dfabbf8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:26.954898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914bea2efd334e6c8dc31698dfabbf8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:26.964112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914bea2efd334e6c8dc31698dfabbf8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:26.964840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914bea2efd334e6c8dc31698dfabbf8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:26.988044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-9d3ee2aa-2555-4533-87f9-5c2ba2280ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914bea2efd334e6c8dc31698dfabbf8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:26.999146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:27.005053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1122/2224] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:26 2022] DELETE /identity/v3/users/914bea2efd334e6c8dc31698dfabbf8e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:27.299539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:27.312484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8fb80-c68e-445c-b3e9-901f127d6bd5 service placement] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:27.364130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:27.434801 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:27.435479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:27.435763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:27.436042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] PATH_INFO: `/v3/projects/caf4c294b65d453d8d0b9cd6fcd76976` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:27.438331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=caf4c294b65d453d8d0b9cd6fcd76976)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:27.441811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:27.521436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caf4c294b65d453d8d0b9cd6fcd76976', 'request_id': 'req-fe03dc19-e9dd-4ac3-b32c-975004e58d41'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:27.534299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caf4c294b65d453d8d0b9cd6fcd76976', 'request_id': 'req-fe03dc19-e9dd-4ac3-b32c-975004e58d41'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:27.541632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caf4c294b65d453d8d0b9cd6fcd76976', 'request_id': 'req-fe03dc19-e9dd-4ac3-b32c-975004e58d41'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:27.573144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] The token cache is being invalidate because project caf4c294b65d453d8d0b9cd6fcd76976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:27.574983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:27.577941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fe03dc19-e9dd-4ac3-b32c-975004e58d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:27.581002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1103/2225] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:27 2022] DELETE /identity/v3/projects/caf4c294b65d453d8d0b9cd6fcd76976 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:27.587991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:27.588610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:27.589102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:27.636785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:27.638354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:27.750083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1123/2226] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:27.758054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:27.758621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:27.758621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:27.781153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:27.782449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:27.792175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1104/2227] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:27.796983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:27.809653 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b0b4b-a39a-4702-b73e-008a611bfaa8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:27.814385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:27.814991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:27.815267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:27.815535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:27.816180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:27.816481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:27.818205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:27.831217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1124/2228] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:47:27.836459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:27.847437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031639a1-d2f7-46e1-bb02-6a4128ca61f0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:27.853757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:27.854436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:27.854725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:27.854997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:27.855697 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:27.857653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:27.885515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1105/2229] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:27 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:27.890324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:27.900363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2b5767-5cfe-47a0-b979-0f2477172827 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:27.962188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:27.962799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:27.963103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:27.963376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:27.963996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:27.966394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:27.988992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1125/2230] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:27 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:27.993263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.005946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62251ce-5236-4463-bdfe-a761399b715f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.011395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.012026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.012302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.012572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.013209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.014873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.029465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1106/2231] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:28.030843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.041096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19be715-b061-46a6-a59b-6f9e3a287669 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.046623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed38d4b0-9f13-44b7-ba3f-bd93001969de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.047231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38d4b0-9f13-44b7-ba3f-bd93001969de admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.047518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38d4b0-9f13-44b7-ba3f-bd93001969de admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.047790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed38d4b0-9f13-44b7-ba3f-bd93001969de admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.048426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38d4b0-9f13-44b7-ba3f-bd93001969de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.050070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed38d4b0-9f13-44b7-ba3f-bd93001969de admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.063322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1126/2232] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:28.067793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.078354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6afd0c4-c824-4bf4-b39e-dcc2fec69c3c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.083186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.083832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.084115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.084395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] PATH_INFO: `/v3/projects/7108a98497ab4470aa865bcb76bcdab5/users/e0abe95a2c5048e2b2b05648a5120b8c/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.098919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7108a98497ab4470aa865bcb76bcdab5, user_id=e0abe95a2c5048e2b2b05648a5120b8c, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.101818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.117060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1107/2233] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:28 2022] PUT /identity/v3/projects/7108a98497ab4470aa865bcb76bcdab5/users/e0abe95a2c5048e2b2b05648a5120b8c/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:28.127824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.128106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.128379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.166212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] MFA Rules not processed for user `e0abe95a2c5048e2b2b05648a5120b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:28.167545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:28.287764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:28.314969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1127/2234] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:28.329148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.338860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:28.446358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767e347-6235-42b7-90b1-4a0470053787 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.452078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3666568-7e35-4559-bf27-996565661323 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.452803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3666568-7e35-4559-bf27-996565661323 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.453187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3666568-7e35-4559-bf27-996565661323 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.454218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3666568-7e35-4559-bf27-996565661323 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.465246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3666568-7e35-4559-bf27-996565661323 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.467070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3666568-7e35-4559-bf27-996565661323 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.473388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:28.488304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1108/2235] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:28.518613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.608072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4662ac7-e2f8-41ae-b73c-2335cf87b061 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.613026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.614095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.614279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.614602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] PATH_INFO: `/v3/users/e0abe95a2c5048e2b2b05648a5120b8c` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.616330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0abe95a2c5048e2b2b05648a5120b8c)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.618325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.686505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0abe95a2c5048e2b2b05648a5120b8c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:28.688049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0abe95a2c5048e2b2b05648a5120b8c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:28.700431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0abe95a2c5048e2b2b05648a5120b8c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:28.701619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0abe95a2c5048e2b2b05648a5120b8c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:28.721524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0abe95a2c5048e2b2b05648a5120b8c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:28.746885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1128/2236] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:28 2022] DELETE /identity/v3/users/e0abe95a2c5048e2b2b05648a5120b8c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:28.752165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.752460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.752739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.779049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:28.782545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:28.859746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1109/2237] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:28.868319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:28.869913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:28.938744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702e498-ee85-4570-959c-1dc0d0f162f7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:28.942305 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c93ed8-dfbd-4fee-a4af-ed79a537cb88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:28.942915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93ed8-dfbd-4fee-a4af-ed79a537cb88 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:28.943193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93ed8-dfbd-4fee-a4af-ed79a537cb88 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:28.943462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c93ed8-dfbd-4fee-a4af-ed79a537cb88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:28.952359 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c93ed8-dfbd-4fee-a4af-ed79a537cb88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:28.953799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c93ed8-dfbd-4fee-a4af-ed79a537cb88 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:28.963726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1129/2238] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:29.360764 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.362384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.384828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bb29a3-44ef-4e23-ad33-b3a01fbe24c5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.404065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.404977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.405269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.405549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] PATH_INFO: `/v3/projects/7108a98497ab4470aa865bcb76bcdab5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.407041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7108a98497ab4470aa865bcb76bcdab5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.408952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:29.481242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7108a98497ab4470aa865bcb76bcdab5', 'request_id': 'req-49930c8d-05bb-4eae-abed-d1c44d23fc4c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:29.488775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7108a98497ab4470aa865bcb76bcdab5', 'request_id': 'req-49930c8d-05bb-4eae-abed-d1c44d23fc4c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:29.500011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7108a98497ab4470aa865bcb76bcdab5', 'request_id': 'req-49930c8d-05bb-4eae-abed-d1c44d23fc4c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:29.520163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] The token cache is being invalidate because project 7108a98497ab4470aa865bcb76bcdab5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:29.520555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:29.521333 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-49930c8d-05bb-4eae-abed-d1c44d23fc4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:29.527218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1110/2239] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:29 2022] DELETE /identity/v3/projects/7108a98497ab4470aa865bcb76bcdab5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:29.528513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8ae8a-99e7-4bdd-a3d6-1a79858afdf5 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.528796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8ae8a-99e7-4bdd-a3d6-1a79858afdf5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.529090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc8ae8a-99e7-4bdd-a3d6-1a79858afdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.552069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2dc8ae8a-99e7-4bdd-a3d6-1a79858afdf5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:29.553328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2dc8ae8a-99e7-4bdd-a3d6-1a79858afdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.571694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:29.630888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.632855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.635037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.668092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:29.669474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.673664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1130/2240] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:29.682899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.685075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.685510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.710356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:29.711599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.721370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1131/2241] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:29.726174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.736518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21fde1-ac7a-4fb1-87f9-0497eb49626b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.741853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.742616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.743038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.743427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.744222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:29.744594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.746468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1111/2242] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:47:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:29.746590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:29.759906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1132/2243] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:47:29.765120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.780725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb78b9-31f7-4f86-a977-4a4c650026da None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.799687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.803041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.803458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.803949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.805358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.806153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.807403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:29.808005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:29.843147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1112/2244] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:29 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:29.847138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.860464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:29.908269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ed1085-9c59-46dc-a8b0-f9600b7e0bce admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.923784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19466346-c93c-4787-8045-0c998ca49766 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.928848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.929448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.929786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.930058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.930781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.932627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:29.959094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1113/2245] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:29 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:29.968512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:29.983574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9266095-858c-4ec0-89f8-5c2f9063118b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:29.988471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:29.989039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:29.989319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:29.989618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:29.990270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:29.992203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.001679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd05646-7c49-4d52-aa6c-29f66211c36e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.002306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd05646-7c49-4d52-aa6c-29f66211c36e service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.002595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd05646-7c49-4d52-aa6c-29f66211c36e service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.002868 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd05646-7c49-4d52-aa6c-29f66211c36e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.007535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1114/2246] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:30.008205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.019111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62321557-b091-4f1c-bdb6-edc6d06aa5d3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.026648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.027290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.027921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.028224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.029417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.031113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.042831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1115/2247] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:30.047487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.053408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:30.060618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617b0b6-372d-4130-820d-e3391750c977 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.073655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.073655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.073655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.073655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] PATH_INFO: `/v3/projects/9fad36a60bc14c3882c31d08a1d87a59/users/4456f8d1c89e4456b864ce858cf8d261/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.084975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd05646-7c49-4d52-aa6c-29f66211c36e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.087430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd05646-7c49-4d52-aa6c-29f66211c36e service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.092320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fad36a60bc14c3882c31d08a1d87a59, user_id=4456f8d1c89e4456b864ce858cf8d261, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.095895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.112503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1133/2248] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:47:30.122727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1116/2249] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:30 2022] PUT /identity/v3/projects/9fad36a60bc14c3882c31d08a1d87a59/users/4456f8d1c89e4456b864ce858cf8d261/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:30.133196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.133487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.133795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.179481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] MFA Rules not processed for user `4456f8d1c89e4456b864ce858cf8d261`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:30.180828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:30.262151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1134/2250] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:30.267699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.337871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a66982-0848-4e1e-819e-f5ff2a2e22c2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.343322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.344104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.344233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.344469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.345097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.347052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.377458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1117/2251] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:30.379671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.390953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2e91f-b234-498d-b6c2-7135a7a1d209 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.450668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.451266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.451566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.451839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.452483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.454440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.482582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1135/2252] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:30.486921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.499105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33808f7-e8b3-49eb-a667-f97779da7a1c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.504125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.508433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.508806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.509160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.513523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.513523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.526613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1118/2253] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:30.531065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.542197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134aff-af22-4e80-a0ba-af0492d97f5e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.547567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e4845d-c512-4b41-9303-e542c902eea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.548178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4845d-c512-4b41-9303-e542c902eea8 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.548462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4845d-c512-4b41-9303-e542c902eea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.548736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e4845d-c512-4b41-9303-e542c902eea8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.549972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4845d-c512-4b41-9303-e542c902eea8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.551252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e4845d-c512-4b41-9303-e542c902eea8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.562118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1136/2254] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:30.566491 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.581844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a1e7a5-ab06-4abc-98d3-1e740661720f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.587108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98ecdc-a375-4481-8f6e-2ccbe37fa151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.587869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ecdc-a375-4481-8f6e-2ccbe37fa151 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.588048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ecdc-a375-4481-8f6e-2ccbe37fa151 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.588317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98ecdc-a375-4481-8f6e-2ccbe37fa151 admin admin] PATH_INFO: `/v3/projects/e8fbecb4852a489a8b0f4f900ea9064b/users/eb2ccf5fd7ec42d7b5d9c250f066ed53/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.610902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ecdc-a375-4481-8f6e-2ccbe37fa151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8fbecb4852a489a8b0f4f900ea9064b, user_id=eb2ccf5fd7ec42d7b5d9c250f066ed53, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.613860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98ecdc-a375-4481-8f6e-2ccbe37fa151 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.629758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:30.639433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1119/2255] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:30 2022] PUT /identity/v3/projects/e8fbecb4852a489a8b0f4f900ea9064b/users/eb2ccf5fd7ec42d7b5d9c250f066ed53/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:30.649264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.649554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.649861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.679045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] MFA Rules not processed for user `eb2ccf5fd7ec42d7b5d9c250f066ed53`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:30.680688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:30.729485 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:30.733863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.734506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.734991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.762171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1137/2256] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:30.764219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:30.765826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:30.766527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.840053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c8971-92d7-4d1b-943c-144b523fcf88 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.845897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.846758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.847040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.847348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.847923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:30.848977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1120/2257] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:30.849913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:30.862979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.888353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1138/2258] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:30.893196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:30.967619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a2d5e-7885-4085-87d4-1f676caa5197 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.988024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4e6b5-1b33-488a-8492-e3c9e4d7a85e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:30.994384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:30.995173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:30.995517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:30.996907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:30.998262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.001379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.028513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1139/2259] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:30 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:31.032382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:31.042451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6828a4-e4fe-4d2b-8aed-46ea930323df admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:31.047153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.048082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.048316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.048596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.048866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.049328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.050112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.051038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.051149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.051662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.065609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1140/2260] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:47:31.068676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:31.080033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9777a-cce9-4e03-9fcc-8fbd96c769b3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:31.085215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.085950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.086236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.086519 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] PATH_INFO: `/v3/projects/77d8989a9e8940d59b822b35ddc62a17/users/72a1c324927e4f2bab16d7969aa5bb8f/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.111590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d8989a9e8940d59b822b35ddc62a17, user_id=72a1c324927e4f2bab16d7969aa5bb8f, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.114469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.116088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:31.129603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1141/2261] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:31 2022] PUT /identity/v3/projects/77d8989a9e8940d59b822b35ddc62a17/users/72a1c324927e4f2bab16d7969aa5bb8f/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:31.129879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.131666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.133459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:31.148334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1121/2262] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:47:31.163322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:31.228513 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41941d-f990-4fd5-8a95-fe9d9f26a7a4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:31.234920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d042c9bd-744e-4e42-be2f-4e945df8c26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.235538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c9bd-744e-4e42-be2f-4e945df8c26d admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.235813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c9bd-744e-4e42-be2f-4e945df8c26d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.236080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042c9bd-744e-4e42-be2f-4e945df8c26d admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.236723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c9bd-744e-4e42-be2f-4e945df8c26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.238354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d042c9bd-744e-4e42-be2f-4e945df8c26d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.257686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1142/2263] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:31 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:31.259223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:31.271181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c587c1-2a3a-4960-9602-1e20a0d8b7fe service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:31.288112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea4ca5-9cf2-47a8-8509-e2bcc3ece723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.288875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4ca5-9cf2-47a8-8509-e2bcc3ece723 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.289231 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4ca5-9cf2-47a8-8509-e2bcc3ece723 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.289595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea4ca5-9cf2-47a8-8509-e2bcc3ece723 admin admin] PATH_INFO: `/v3/projects/77d8989a9e8940d59b822b35ddc62a17/users/72a1c324927e4f2bab16d7969aa5bb8f/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.291659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea4ca5-9cf2-47a8-8509-e2bcc3ece723 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d8989a9e8940d59b822b35ddc62a17, user_id=72a1c324927e4f2bab16d7969aa5bb8f, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.295057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea4ca5-9cf2-47a8-8509-e2bcc3ece723 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.306890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1122/2264] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:31 2022] PUT /identity/v3/projects/77d8989a9e8940d59b822b35ddc62a17/users/72a1c324927e4f2bab16d7969aa5bb8f/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:31.318037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.318676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.318954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.359464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] MFA Rules not processed for user `72a1c324927e4f2bab16d7969aa5bb8f`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:31.361415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:31.510285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.510285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.510285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.517060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1143/2265] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:31.530269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:31.532743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:31.556058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:31.557340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:31.606829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563b647-b765-4b99-9fc7-6f94aa2477ac None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:31.614274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61b824-6ee0-4adc-bf28-ea209a21ad9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.615607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61b824-6ee0-4adc-bf28-ea209a21ad9d service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.616053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61b824-6ee0-4adc-bf28-ea209a21ad9d service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.616428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61b824-6ee0-4adc-bf28-ea209a21ad9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.624461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1123/2266] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:47:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:31.641015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:31.684769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:31.690079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61b824-6ee0-4adc-bf28-ea209a21ad9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.691915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61b824-6ee0-4adc-bf28-ea209a21ad9d service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.704032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1144/2267] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:31.738913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d612c0-5608-43c1-8591-4ba55333b87f None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:31.744469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:31.745310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:31.745726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:31.746208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:31.759314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:31.762040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:31.778971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1124/2268] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:47:32.263387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:32.263841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:32.263926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:32.283950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:32.285087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:32.292273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1145/2269] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 05:47:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:32.305069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:32.315659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac6931-74d0-4394-a467-767378dcdeeb service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:32.320728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:32.321319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:32.321628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:32.321900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:32.329802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:32.331366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:32.339202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1125/2270] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:47:33.487841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:33.489968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:33.509903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1ce3aa-4274-49fb-90bc-42403ad0e9d7 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:33.526614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:33.528384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:33.528875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:33.529400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:33.602627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:33.604964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:33.622429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1146/2271] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:34.349748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:34.360717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24adbbd5-2306-4390-9f88-4ebf79f3ba6f service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:34.380395 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46289e2b-5574-4c3d-b30f-9d1c50c65cf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:34.381302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289e2b-5574-4c3d-b30f-9d1c50c65cf8 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:34.381888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289e2b-5574-4c3d-b30f-9d1c50c65cf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:34.382341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46289e2b-5574-4c3d-b30f-9d1c50c65cf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:34.397971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289e2b-5574-4c3d-b30f-9d1c50c65cf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:34.399668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46289e2b-5574-4c3d-b30f-9d1c50c65cf8 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:34.411470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1126/2272] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:34 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:34.567388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:34.570680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:34.660054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d72f01-5f7a-45aa-a870-25798bec5ca2 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:34.674354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:34.675066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:34.675409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:34.675749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] PATH_INFO: `/v3/users/2c9fa86b65ec431eb3f77cf2bfe5cda4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:34.676996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c9fa86b65ec431eb3f77cf2bfe5cda4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:34.679774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:34.730091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c9fa86b65ec431eb3f77cf2bfe5cda4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.730907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c9fa86b65ec431eb3f77cf2bfe5cda4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.740830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c9fa86b65ec431eb3f77cf2bfe5cda4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.741730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c9fa86b65ec431eb3f77cf2bfe5cda4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.760404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c9fa86b65ec431eb3f77cf2bfe5cda4'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.784224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1147/2273] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:34 2022] DELETE /identity/v3/users/2c9fa86b65ec431eb3f77cf2bfe5cda4 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:34.788919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:34.789371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:34.789799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:34.809703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:34.811018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:34.854847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:34.876046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1127/2274] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:34.887548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:34.888137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:34.889728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:34.889885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:34.896814 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:34.906302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcaab0f-92a1-4f3b-b8d3-2139fc08960f None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:34.912421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:34.913063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:34.913638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:34.914074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] PATH_INFO: `/v3/users/8c5da033360f4bba808bfdfdef2b0ba3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:34.915295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c5da033360f4bba808bfdfdef2b0ba3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:34.917299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:34.961162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff506719-5169-4a30-b658-af0b8d05f655 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:34.965543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:34.966181 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:34.966463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:34.966737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:34.972065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c5da033360f4bba808bfdfdef2b0ba3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.972906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c5da033360f4bba808bfdfdef2b0ba3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.976082 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:34.977546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:34.979930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c5da033360f4bba808bfdfdef2b0ba3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.981126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c5da033360f4bba808bfdfdef2b0ba3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:34.988136 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1148/2275] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:47:34.988845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c5da033360f4bba808bfdfdef2b0ba3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.008051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1128/2276] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:34 2022] DELETE /identity/v3/users/8c5da033360f4bba808bfdfdef2b0ba3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:35.178490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:35.180116 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:35.195321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5f8c-9ad2-47be-bed8-3b66435b98de service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:35.225744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:35.227468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:35.278014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:35.278734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:35.279096 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:35.279374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] PATH_INFO: `/v3/projects/873d8330010249f58ac18c8e17946f6e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:35.282307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=873d8330010249f58ac18c8e17946f6e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:35.284296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:35.310176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f7b61-1071-4ad6-9066-62086bae72b0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:35.316506 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:35.317304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:35.317710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:35.317989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] PATH_INFO: `/v3/projects/33a4f0e0d56743d587f90e129055dc06` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:35.319136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33a4f0e0d56743d587f90e129055dc06)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:35.321051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:35.335672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '873d8330010249f58ac18c8e17946f6e', 'request_id': 'req-16421e5a-aaab-4863-be20-c5270c4a6e8d'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.341897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '873d8330010249f58ac18c8e17946f6e', 'request_id': 'req-16421e5a-aaab-4863-be20-c5270c4a6e8d'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.350361 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '873d8330010249f58ac18c8e17946f6e', 'request_id': 'req-16421e5a-aaab-4863-be20-c5270c4a6e8d'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.363360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] The token cache is being invalidate because project 873d8330010249f58ac18c8e17946f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:35.363648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.364282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.365791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1149/2277] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:35 2022] DELETE /identity/v3/projects/873d8330010249f58ac18c8e17946f6e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:35.381881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33a4f0e0d56743d587f90e129055dc06', 'request_id': 'req-62116e30-13cd-47eb-a183-6c910dcb1136'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.389325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33a4f0e0d56743d587f90e129055dc06', 'request_id': 'req-62116e30-13cd-47eb-a183-6c910dcb1136'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.390892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:35.398098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33a4f0e0d56743d587f90e129055dc06', 'request_id': 'req-62116e30-13cd-47eb-a183-6c910dcb1136'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.423771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] The token cache is being invalidate because project 33a4f0e0d56743d587f90e129055dc06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:35.424153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.425109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.430862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1129/2278] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:35 2022] DELETE /identity/v3/projects/33a4f0e0d56743d587f90e129055dc06 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:35.432448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:35.439337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:35.498766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:35.554780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16421e5a-aaab-4863-be20-c5270c4a6e8d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:35.651736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:35.653042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:35.653236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:35.653504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] PATH_INFO: `/v3/users/4f0a8e275cdb45a393b63cc4e1040a58` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:35.655369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f0a8e275cdb45a393b63cc4e1040a58)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:35.657901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:35.737288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f0a8e275cdb45a393b63cc4e1040a58'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.738055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f0a8e275cdb45a393b63cc4e1040a58'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.756678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f0a8e275cdb45a393b63cc4e1040a58'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.757613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f0a8e275cdb45a393b63cc4e1040a58'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.773235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f0a8e275cdb45a393b63cc4e1040a58'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:35.790122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1150/2279] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:35 2022] DELETE /identity/v3/users/4f0a8e275cdb45a393b63cc4e1040a58 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:36.030997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:36.032663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:36.050078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62116e30-13cd-47eb-a183-6c910dcb1136 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:36.110322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:36.111044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:36.111328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:36.111840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] PATH_INFO: `/v3/projects/d34e44bd4f65471b9c8560eee7fd26dc` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:36.113034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34e44bd4f65471b9c8560eee7fd26dc)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:36.115124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:36.172670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34e44bd4f65471b9c8560eee7fd26dc', 'request_id': 'req-7ec0130c-1fdd-4172-9dd3-0132195e632d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:36.180349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34e44bd4f65471b9c8560eee7fd26dc', 'request_id': 'req-7ec0130c-1fdd-4172-9dd3-0132195e632d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:36.193808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34e44bd4f65471b9c8560eee7fd26dc', 'request_id': 'req-7ec0130c-1fdd-4172-9dd3-0132195e632d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:36.219245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] The token cache is being invalidate because project d34e44bd4f65471b9c8560eee7fd26dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:36.219533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:36.220147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ec0130c-1fdd-4172-9dd3-0132195e632d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:36.221784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1130/2280] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:36 2022] DELETE /identity/v3/projects/d34e44bd4f65471b9c8560eee7fd26dc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:38.490979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:38.492523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:38.508222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3a2b8-d3a9-4592-aff3-81795732a380 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:38.578804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:38.581193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:38.581193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:38.581193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] PATH_INFO: `/v3/users/4456f8d1c89e4456b864ce858cf8d261` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:38.582618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4456f8d1c89e4456b864ce858cf8d261)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:38.584629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:38.592860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:38.642650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4456f8d1c89e4456b864ce858cf8d261'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:38.643296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4456f8d1c89e4456b864ce858cf8d261'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:38.650258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4456f8d1c89e4456b864ce858cf8d261'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:38.651182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4456f8d1c89e4456b864ce858cf8d261'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:38.665398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4456f8d1c89e4456b864ce858cf8d261'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:38.678162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1151/2281] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:38 2022] DELETE /identity/v3/users/4456f8d1c89e4456b864ce858cf8d261 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:38.682724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:38.683208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:38.683301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:38.703719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:38.704962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:38.765947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1131/2282] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:38.775794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:38.777207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:38.834220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ee200a-1de5-4942-907a-33b39f70e36e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:38.837903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:38.838506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:38.838787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:38.839058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:38.848418 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:38.849879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:38.860023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1152/2283] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:39.137006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:39.138983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:39.153472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef47d6-6b9e-4738-8278-1104aa1d5fae None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:39.160097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:39.160829 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:39.161040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:39.161392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] PATH_INFO: `/v3/projects/9fad36a60bc14c3882c31d08a1d87a59` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:39.162660 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fad36a60bc14c3882c31d08a1d87a59)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:39.164573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:39.176750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:39.220535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fad36a60bc14c3882c31d08a1d87a59', 'request_id': 'req-0ec56736-b8c3-46d7-b824-b1e58a63f158'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.226004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fad36a60bc14c3882c31d08a1d87a59', 'request_id': 'req-0ec56736-b8c3-46d7-b824-b1e58a63f158'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.232787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fad36a60bc14c3882c31d08a1d87a59', 'request_id': 'req-0ec56736-b8c3-46d7-b824-b1e58a63f158'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.245769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] The token cache is being invalidate because project 9fad36a60bc14c3882c31d08a1d87a59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:39.246057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.247373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.249045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1132/2284] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:39 2022] DELETE /identity/v3/projects/9fad36a60bc14c3882c31d08a1d87a59 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:39.252939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:39.263173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc0d38-7e39-48dd-922a-cb55be73ddeb service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:39.323580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:39.324213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:39.324531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:39.324828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] PATH_INFO: `/v3/users/eb2ccf5fd7ec42d7b5d9c250f066ed53` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:39.325960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb2ccf5fd7ec42d7b5d9c250f066ed53)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:39.327893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:39.359134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:39.385472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb2ccf5fd7ec42d7b5d9c250f066ed53'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.386177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb2ccf5fd7ec42d7b5d9c250f066ed53'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.394830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb2ccf5fd7ec42d7b5d9c250f066ed53'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.395834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb2ccf5fd7ec42d7b5d9c250f066ed53'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.407959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb2ccf5fd7ec42d7b5d9c250f066ed53'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.420381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1153/2285] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:39 2022] DELETE /identity/v3/users/eb2ccf5fd7ec42d7b5d9c250f066ed53 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:39.787742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:39.789812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:39.829042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec56736-b8c3-46d7-b824-b1e58a63f158 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:39.913335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:39.914719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:39.915180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:39.915618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] PATH_INFO: `/v3/projects/e8fbecb4852a489a8b0f4f900ea9064b` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:39.917118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8fbecb4852a489a8b0f4f900ea9064b)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:39.920861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:39.988543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8fbecb4852a489a8b0f4f900ea9064b', 'request_id': 'req-fea34933-703d-4dcc-82f1-8f7e48cfc04f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:39.994596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8fbecb4852a489a8b0f4f900ea9064b', 'request_id': 'req-fea34933-703d-4dcc-82f1-8f7e48cfc04f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.003232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8fbecb4852a489a8b0f4f900ea9064b', 'request_id': 'req-fea34933-703d-4dcc-82f1-8f7e48cfc04f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.018707 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] The token cache is being invalidate because project e8fbecb4852a489a8b0f4f900ea9064b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:40.019039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.019738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.021477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1133/2286] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:39 2022] DELETE /identity/v3/projects/e8fbecb4852a489a8b0f4f900ea9064b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:40.025313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:40.026930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:40.040510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074de381-2c19-4eca-b8d1-028688b0007a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:40.119232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:40.119878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:40.120165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:40.120450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] PATH_INFO: `/v3/users/72a1c324927e4f2bab16d7969aa5bb8f` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:40.122367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a1c324927e4f2bab16d7969aa5bb8f)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:40.124248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:40.191546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a1c324927e4f2bab16d7969aa5bb8f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.192213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a1c324927e4f2bab16d7969aa5bb8f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.202111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a1c324927e4f2bab16d7969aa5bb8f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.202769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a1c324927e4f2bab16d7969aa5bb8f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.218330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a67c0287-65a5-48cd-8308-84d8f567dfeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a1c324927e4f2bab16d7969aa5bb8f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.234276 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1154/2287] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:47:40 2022] DELETE /identity/v3/users/72a1c324927e4f2bab16d7969aa5bb8f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:40.546467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:40.551205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:40.582962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea34933-703d-4dcc-82f1-8f7e48cfc04f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:40.618867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:40.655644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:40.656298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:40.656580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:40.656857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] PATH_INFO: `/v3/projects/77d8989a9e8940d59b822b35ddc62a17` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:40.658030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d8989a9e8940d59b822b35ddc62a17)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:40.659970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:40.708352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d8989a9e8940d59b822b35ddc62a17', 'request_id': 'req-5a469c97-83eb-4166-9c26-e2314f8fff71'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.715027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d8989a9e8940d59b822b35ddc62a17', 'request_id': 'req-5a469c97-83eb-4166-9c26-e2314f8fff71'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.723783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d8989a9e8940d59b822b35ddc62a17', 'request_id': 'req-5a469c97-83eb-4166-9c26-e2314f8fff71'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.739351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] The token cache is being invalidate because project 77d8989a9e8940d59b822b35ddc62a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:47:40.739647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.740440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5a469c97-83eb-4166-9c26-e2314f8fff71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:47:40.742463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1134/2288] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:47:40 2022] DELETE /identity/v3/projects/77d8989a9e8940d59b822b35ddc62a17 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:47:40.748812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:40.749134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:40.749439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:40.772690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:40.773991 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:40.795543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:40.838807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1155/2289] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:40.847164 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:40.847679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:40.847770 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:40.867942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:40.869258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:40.877535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1135/2290] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:40.882483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:40.893500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06fdc8-09d3-4e26-aaf9-7cc56542942d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:40.898390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:40.899032 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:40.899312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:40.899585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:40.900233 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:47:40.900537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:40.902189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:40.914968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1156/2291] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:47:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:47:40.919846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:40.929708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f526f05d-209b-49c9-b405-34f928d8e42e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:40.935031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:40.935625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:40.935902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:40.936172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:40.936823 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:40.938774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:40.970166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1136/2292] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:47:40 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:47:40.971270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:40.981929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545fca0-69f4-4a7b-b397-6dcb4df83f6d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:41.041782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:41.042495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:41.042659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:41.042931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:41.044122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:41.046109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:41.066615 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1157/2293] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:47:40 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:47:41.070755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:41.082706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e956b0e-0d20-499b-8794-2827437ac13c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:41.088495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:41.089496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:41.090454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:41.090996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:41.091716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:41.093641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:41.104751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1137/2294] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:41.109796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:41.121153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae316436-2187-4bb0-8e4c-55a6813b1aa9 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:41.125958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e564183-2910-4a93-982b-7637329d08d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:41.126554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564183-2910-4a93-982b-7637329d08d4 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:41.126858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564183-2910-4a93-982b-7637329d08d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:41.127128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e564183-2910-4a93-982b-7637329d08d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:41.127763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564183-2910-4a93-982b-7637329d08d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:41.129325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e564183-2910-4a93-982b-7637329d08d4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:41.140441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1158/2295] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:47:41.144573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:41.155118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608f1b1-fe8c-4f46-aa26-d79a3e474eb1 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:41.160210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:41.161091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:41.161091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:41.161795 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] PATH_INFO: `/v3/projects/0adadc58ec4f4132b2887b4c3457c95a/users/b270da20aae941ca8994d930f3e0ab0c/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:41.182373 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0adadc58ec4f4132b2887b4c3457c95a, user_id=b270da20aae941ca8994d930f3e0ab0c, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:41.185883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:41.200052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1138/2296] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:47:41 2022] PUT /identity/v3/projects/0adadc58ec4f4132b2887b4c3457c95a/users/b270da20aae941ca8994d930f3e0ab0c/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:47:41.211506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a8281-3133-4586-9804-7fec8d9677f2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:41.211800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a8281-3133-4586-9804-7fec8d9677f2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:41.212077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689a8281-3133-4586-9804-7fec8d9677f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:41.239863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-689a8281-3133-4586-9804-7fec8d9677f2 None None] MFA Rules not processed for user `b270da20aae941ca8994d930f3e0ab0c`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:41.241291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-689a8281-3133-4586-9804-7fec8d9677f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:41.311955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1159/2297] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:47:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:47:41.323459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:41.325778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:41.381066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cffbe-19c8-4998-b48c-14b90eb86627 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:41.384685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:41.385277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:41.385553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:41.385849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:41.395068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:41.396514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:41.401206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:41.407142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1139/2298] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:47:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:47:50.523431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e662-cc4d-4ced-a2cf-52680566969d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:50.524080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e662-cc4d-4ced-a2cf-52680566969d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:50.524080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b2e662-cc4d-4ced-a2cf-52680566969d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:50.543423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-85b2e662-cc4d-4ced-a2cf-52680566969d None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:47:50.544671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-85b2e662-cc4d-4ced-a2cf-52680566969d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:50.560871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:47:50.612711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1160/2299] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:47:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:47:50.626377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:47:50.627920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:47:50.710196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7f1a9-2768-4b5e-9192-8dcefdc188ac service glance] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:47:50.715242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:47:50.715791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:47:50.716073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:47:50.716349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:47:50.725237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:47:50.726990 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:47:50.736136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1140/2300] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:47:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:48:00.605412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=16 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:00.732053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=32 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:06.053586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17033b4f-345d-4a8d-89ea-2e6e9c54467f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:06.054720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17033b4f-345d-4a8d-89ea-2e6e9c54467f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:06.055197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17033b4f-345d-4a8d-89ea-2e6e9c54467f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:06.075526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-17033b4f-345d-4a8d-89ea-2e6e9c54467f None None] MFA Rules not processed for user `c8b7cb87296a4ecea0b9ab44a0cbfd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:06.076886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-17033b4f-345d-4a8d-89ea-2e6e9c54467f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:06.156128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1161/2301] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:08.880946 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:08.882740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:09.012529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9a94f-b245-4b67-ba1a-c3ccfb4711ec service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:09.019394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:09.020286 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:09.020769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:09.020997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] PATH_INFO: `/v3/users/2c7d0f3ad00e44c2827553d7f0e73ff3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:09.022208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c7d0f3ad00e44c2827553d7f0e73ff3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:09.025129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:09.108070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c7d0f3ad00e44c2827553d7f0e73ff3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:09.109735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c7d0f3ad00e44c2827553d7f0e73ff3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:09.125611 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c7d0f3ad00e44c2827553d7f0e73ff3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:09.129792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c7d0f3ad00e44c2827553d7f0e73ff3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:09.146689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c7d0f3ad00e44c2827553d7f0e73ff3'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:09.195280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1141/2302] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:08 2022] DELETE /identity/v3/users/2c7d0f3ad00e44c2827553d7f0e73ff3 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:09.197016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:09.197938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:09.198438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:09.230248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:09.236051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:09.365288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1162/2303] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:09.385594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:09.387185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:09.513351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244be60e-d2c4-4e3c-9e4e-c0cce02b0b3a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:09.543271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69082b-0446-4ea5-9088-3d0b5097d325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:09.544372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69082b-0446-4ea5-9088-3d0b5097d325 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:09.545159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69082b-0446-4ea5-9088-3d0b5097d325 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:09.545678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69082b-0446-4ea5-9088-3d0b5097d325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:09.557481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69082b-0446-4ea5-9088-3d0b5097d325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:09.559716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69082b-0446-4ea5-9088-3d0b5097d325 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:09.572532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1142/2304] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:09.993601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:09.997104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:10.040971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8efd81-8abd-41ab-aceb-d72f6a44e0e7 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.051638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.052386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.052699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.052947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] PATH_INFO: `/v3/projects/58b80d12a1534241b8b9e7ebb9950062` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.055280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=58b80d12a1534241b8b9e7ebb9950062)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:10.057816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:10.151691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58b80d12a1534241b8b9e7ebb9950062', 'request_id': 'req-ba367db9-7ff9-4a70-9860-3505f76f3a3f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:10.160499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58b80d12a1534241b8b9e7ebb9950062', 'request_id': 'req-ba367db9-7ff9-4a70-9860-3505f76f3a3f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:10.172902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58b80d12a1534241b8b9e7ebb9950062', 'request_id': 'req-ba367db9-7ff9-4a70-9860-3505f76f3a3f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:10.188533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] The token cache is being invalidate because project 58b80d12a1534241b8b9e7ebb9950062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:10.188832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:10.189514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ba367db9-7ff9-4a70-9860-3505f76f3a3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:10.193834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1163/2305] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:09 2022] DELETE /identity/v3/projects/58b80d12a1534241b8b9e7ebb9950062 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:10.200028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.200531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.201028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.241473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:10.242836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:10.267803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:10.341913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1143/2306] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:10.352050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.352050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.352050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.374618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:10.375883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:10.387036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1164/2307] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:10.389955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:10.401051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387194e6-b397-4efc-8e19-41ce5bcfb2d4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.407447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.410381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.410960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.410960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.411718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:10.411969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:10.413670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:10.437648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1144/2308] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:48:10.442309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:10.457168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f28003-1fba-4214-a50a-02f064e981e3 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.464562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.465384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.465946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.466297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.467285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:10.469443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:10.510056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:10.526244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1165/2309] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:10 2022] POST /identity/v3/projects => generated 381 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:10.530355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:10.545045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ecb8a-a6d2-4e30-859f-34a115e5d138 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.621104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.621850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.622206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.622551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.623254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:10.625257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:10.658296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1145/2310] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:10 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:10.660224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:10.672783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee29306-04d8-477c-a2be-3f2ee11c3aed admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.681046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.681982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.682376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.682654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.683726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:10.685836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:10.701271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1166/2311] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:10.702514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:10.722487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009f6fa4-dd7c-43e4-897c-9d5c71e50ed2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.728298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f833d6-d1ce-4da6-ab3c-847b59eb22c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.729062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f833d6-d1ce-4da6-ab3c-847b59eb22c8 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.729578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f833d6-d1ce-4da6-ab3c-847b59eb22c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.730465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f833d6-d1ce-4da6-ab3c-847b59eb22c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:10.731220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f833d6-d1ce-4da6-ab3c-847b59eb22c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:10.734515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f833d6-d1ce-4da6-ab3c-847b59eb22c8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:10.759746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1146/2312] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:10.763897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:10.784698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37e2c3-0446-4769-8511-eb5f9fb52646 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:10.798499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:10.799266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:10.800197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:10.800579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] PATH_INFO: `/v3/projects/87f7a45fce204c9c9c15fae64b26b61e/users/8e9cefedd37f43e6bb64a270cde9f6b6/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:11.050031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f7a45fce204c9c9c15fae64b26b61e, user_id=8e9cefedd37f43e6bb64a270cde9f6b6, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:11.053175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:11.077139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1167/2313] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:10 2022] PUT /identity/v3/projects/87f7a45fce204c9c9c15fae64b26b61e/users/8e9cefedd37f43e6bb64a270cde9f6b6/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:11.093605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933ac3f-849a-4186-85cf-85d8c192318a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:11.093605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933ac3f-849a-4186-85cf-85d8c192318a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:11.093605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b933ac3f-849a-4186-85cf-85d8c192318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:11.144496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b933ac3f-849a-4186-85cf-85d8c192318a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:11.146080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b933ac3f-849a-4186-85cf-85d8c192318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:11.306241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1147/2314] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:11.353421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:11.356287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:11.543084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb214015-ea42-42ef-9ad7-c1df772e0c6f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:11.547731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:11.548364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:11.548632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:11.548900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:11.559402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:11.561001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:11.574422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1168/2315] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:12.761336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f04da-6c94-4d47-b64b-f6e8591dda21 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:12.761859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f04da-6c94-4d47-b64b-f6e8591dda21 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:12.761939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f04da-6c94-4d47-b64b-f6e8591dda21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:12.830918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-442f04da-6c94-4d47-b64b-f6e8591dda21 None None] MFA Rules not processed for user `8e9cefedd37f43e6bb64a270cde9f6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:12.832737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-442f04da-6c94-4d47-b64b-f6e8591dda21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:12.889422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:13.006789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1148/2316] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:13.019845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:13.025194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:13.192558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a95a3-fe4b-49db-ad76-a38bc9671ab5 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:13.211375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2dd996-f8e2-4fa9-81af-b167bb9dbee2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:13.211375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2dd996-f8e2-4fa9-81af-b167bb9dbee2 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:13.211375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2dd996-f8e2-4fa9-81af-b167bb9dbee2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:13.211375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2dd996-f8e2-4fa9-81af-b167bb9dbee2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:13.232554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2dd996-f8e2-4fa9-81af-b167bb9dbee2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:13.234282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2dd996-f8e2-4fa9-81af-b167bb9dbee2 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:13.258971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1169/2317] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:13 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:17.902284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf0051-c9e5-46ae-8aa4-1493094f704d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:17.902840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf0051-c9e5-46ae-8aa4-1493094f704d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:17.902840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf0051-c9e5-46ae-8aa4-1493094f704d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:17.926813 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-aedf0051-c9e5-46ae-8aa4-1493094f704d None None] MFA Rules not processed for user `8e9cefedd37f43e6bb64a270cde9f6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:17.928210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-aedf0051-c9e5-46ae-8aa4-1493094f704d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:17.941161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1149/2318] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:19.804489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:19.805046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:19.805046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:19.824231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] MFA Rules not processed for user `3d8467cc62d948038690af1fdf5bb96a`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:19.825443 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:19.885811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1170/2319] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:20.319591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:20.320426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:20.320857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:20.341361 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:20.342725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:20.421503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1150/2320] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 05:48:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:20.434890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:20.436267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:20.456670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:20.534710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae6013-cb89-45c8-9bc3-5ca7872bd5c5 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:20.542132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eab3b5-b2f0-4bfc-b6ac-afbeb59f108c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:20.542939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab3b5-b2f0-4bfc-b6ac-afbeb59f108c service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:20.543312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab3b5-b2f0-4bfc-b6ac-afbeb59f108c service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:20.543746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab3b5-b2f0-4bfc-b6ac-afbeb59f108c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:20.557347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eab3b5-b2f0-4bfc-b6ac-afbeb59f108c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:20.559682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eab3b5-b2f0-4bfc-b6ac-afbeb59f108c service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:20.576375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1171/2321] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:48:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:48:22.288036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:22.289786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:22.308196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a80320-95d2-4c77-bee3-8f926c9848e8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:22.316710 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:22.318449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:22.318743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:22.319076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] PATH_INFO: `/v3/users/b270da20aae941ca8994d930f3e0ab0c` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:22.320559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b270da20aae941ca8994d930f3e0ab0c)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:22.324026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:22.409632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b270da20aae941ca8994d930f3e0ab0c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:22.410919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b270da20aae941ca8994d930f3e0ab0c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:22.423536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b270da20aae941ca8994d930f3e0ab0c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:22.424588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b270da20aae941ca8994d930f3e0ab0c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:22.436618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b270da20aae941ca8994d930f3e0ab0c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:22.463883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1151/2322] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:22 2022] DELETE /identity/v3/users/b270da20aae941ca8994d930f3e0ab0c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:22.468382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496513b-6d64-42f5-b99c-2415a51b25d7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:22.468864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496513b-6d64-42f5-b99c-2415a51b25d7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:22.468950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496513b-6d64-42f5-b99c-2415a51b25d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:22.515394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a496513b-6d64-42f5-b99c-2415a51b25d7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:22.526607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a496513b-6d64-42f5-b99c-2415a51b25d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:22.604707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1172/2323] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:22.616272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:22.618122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:22.656058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:22.689824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec0d94-c817-4dca-a2bf-605fdbb87fe2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:22.694100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622653a2-bd91-490b-b595-1214c5b92d89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:22.694855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622653a2-bd91-490b-b595-1214c5b92d89 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:22.695140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622653a2-bd91-490b-b595-1214c5b92d89 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:22.695423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622653a2-bd91-490b-b595-1214c5b92d89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:22.707298 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622653a2-bd91-490b-b595-1214c5b92d89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:22.708762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622653a2-bd91-490b-b595-1214c5b92d89 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:22.719828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1152/2324] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:23.811007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd4e1-4c87-4f8c-971d-d33f39df3983 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:23.811893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd4e1-4c87-4f8c-971d-d33f39df3983 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:23.812447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49fd4e1-4c87-4f8c-971d-d33f39df3983 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:23.813980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1173/2325] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:48:23.849737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:23.850875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:23.851216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:23.878868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] MFA Rules not processed for user `61a7bc1bc86a405a9ef04a3d36502a7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:23.880217 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:23.976367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1153/2326] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:48:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:23.999719 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd4e1-4c87-4f8c-971d-d33f39df3983 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.001465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f49fd4e1-4c87-4f8c-971d-d33f39df3983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.020975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49fd4e1-4c87-4f8c-971d-d33f39df3983 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.026521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.027139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.027595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.028012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.039122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.040740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.052556 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.056460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.095206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1174/2327] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:24.108243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.109623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985035-c9bb-4047-b706-8f710db07d23 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.119303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.123176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.123996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.124282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.124563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] PATH_INFO: `/v3/users/a621860d8fcb4327820ce6b802e6a725` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.132309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a621860d8fcb4327820ce6b802e6a725)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.134415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.159620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68ac53b-cf1c-4bd4-8a16-1f6dfdd2382b service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.166003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.166752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.167038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.167318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] PATH_INFO: `/v3/projects/0adadc58ec4f4132b2887b4c3457c95a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.168690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0adadc58ec4f4132b2887b4c3457c95a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.171735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.203686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a621860d8fcb4327820ce6b802e6a725'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.204388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a621860d8fcb4327820ce6b802e6a725'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.214718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a621860d8fcb4327820ce6b802e6a725'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.215688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a621860d8fcb4327820ce6b802e6a725'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.226835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-408041a8-77e4-4eed-aaf8-2431b40b72e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a621860d8fcb4327820ce6b802e6a725'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.228873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0adadc58ec4f4132b2887b4c3457c95a', 'request_id': 'req-c1ce4776-c970-4ef6-ad99-07d9b08cb437'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.235243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0adadc58ec4f4132b2887b4c3457c95a', 'request_id': 'req-c1ce4776-c970-4ef6-ad99-07d9b08cb437'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.250916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0adadc58ec4f4132b2887b4c3457c95a', 'request_id': 'req-c1ce4776-c970-4ef6-ad99-07d9b08cb437'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.271862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] The token cache is being invalidate because project 0adadc58ec4f4132b2887b4c3457c95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:24.272153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.273008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c1ce4776-c970-4ef6-ad99-07d9b08cb437 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.278778 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1154/2329] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:48:24 2022] DELETE /identity/v3/users/a621860d8fcb4327820ce6b802e6a725 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:24.279314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1175/2329] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:24 2022] DELETE /identity/v3/projects/0adadc58ec4f4132b2887b4c3457c95a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:24.279895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.280181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.280461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.305015 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:24.306156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.350372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:24.394932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1155/2330] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:24.397842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.398012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.398182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.434370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:24.435730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.447875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1176/2331] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:24.452378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.466688 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f50bd6a-cbfa-4814-a614-5dc97ffc9a3d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.480595 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.482720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.483805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.484067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.484322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.485236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.485236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:24.485553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.487459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.511114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1156/2332] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:48:24.526651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.541199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b7c55-3b71-4d5f-938b-3f0b7a0bb6f1 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.548285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.548872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.549715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.550026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] PATH_INFO: `/v3/projects/82f6c80e9245427aad8acd3c8bd5efbc` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.551381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f6c80e9245427aad8acd3c8bd5efbc)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.553931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.557077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e1da0-a444-47fb-b00c-224922d21a1c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.560219 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:24.570218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.573067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.573492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.573875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.574486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.579031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.624783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1157/2333] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:24.628530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.646581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f6c80e9245427aad8acd3c8bd5efbc', 'request_id': 'req-8debe418-6c0d-446a-8127-775b4eaf5900'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.658945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f6c80e9245427aad8acd3c8bd5efbc', 'request_id': 'req-8debe418-6c0d-446a-8127-775b4eaf5900'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.671078 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f6c80e9245427aad8acd3c8bd5efbc', 'request_id': 'req-8debe418-6c0d-446a-8127-775b4eaf5900'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.686665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] The token cache is being invalidate because project 82f6c80e9245427aad8acd3c8bd5efbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:48:24.686900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.688380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8debe418-6c0d-446a-8127-775b4eaf5900 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:48:24.692414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1177/2334] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:48:24 2022] DELETE /identity/v3/projects/82f6c80e9245427aad8acd3c8bd5efbc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:24.698077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b916d2-ff30-478c-8265-11a1a8f14133 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.698496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b916d2-ff30-478c-8265-11a1a8f14133 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.698794 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b916d2-ff30-478c-8265-11a1a8f14133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.719678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-62b916d2-ff30-478c-8265-11a1a8f14133 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:24.720907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-62b916d2-ff30-478c-8265-11a1a8f14133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.728894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557f334-c561-4744-94ca-c95f81cde79b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.786800 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1178/2335] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:24.794197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.794478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.794758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.804402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.805101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.805429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.805737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.806383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.808419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.813845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:24.815095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.823630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1179/2336] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:24.829322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.838115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1158/2337] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/users => generated 287 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:24.838507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a458b-5b58-4bb8-8522-681bc982196e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.839168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.840983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:24.843199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.843801 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.844079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.844348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.844988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:48:24.845285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.846936 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.852708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fbf681-32e4-4ae3-a40d-e3e5c5bb2239 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.857623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.858214 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.858757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.859027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.859661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.861309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.863062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1180/2338] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:48:24.863850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.875814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1159/2339] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:24.876000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f20e0c-e121-4d2e-b85f-16c6c0ebdfe4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.877174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.878884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.879498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.879777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.880068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.880669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.882604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.889637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef44ca8-bd50-4c3a-88f1-070300359446 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.894526 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:24.895195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:24.895540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:24.895881 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:24.896631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:24.898334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:24.910675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1160/2340] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:48:24.914921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.916094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1181/2341] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:24.916988 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:24.932039 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2dbfd6-b46d-46b2-9956-924b5ab93ced admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:24.951220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:24.990518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:25.006517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090dd99-70d8-4b20-b54b-606fadefe8b4 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:25.010993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d78dba-d782-4f9e-b286-322aede1b408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.011785 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78dba-d782-4f9e-b286-322aede1b408 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.012095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78dba-d782-4f9e-b286-322aede1b408 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.012571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d78dba-d782-4f9e-b286-322aede1b408 admin admin] PATH_INFO: `/v3/projects/1406fa515f224f4b8507331fd22ea4fd/users/135e823ff7f64beab0689c3a84cc268f/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.014660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.015475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.015840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.016186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.017343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.020051 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.033318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d78dba-d782-4f9e-b286-322aede1b408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1406fa515f224f4b8507331fd22ea4fd, user_id=135e823ff7f64beab0689c3a84cc268f, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.036220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d78dba-d782-4f9e-b286-322aede1b408 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.053194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1182/2342] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:24 2022] POST /identity/v3/users => generated 292 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:25.057664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1161/2343] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:24 2022] PUT /identity/v3/projects/1406fa515f224f4b8507331fd22ea4fd/users/135e823ff7f64beab0689c3a84cc268f/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:25.059190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:25.063686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.064204 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.064639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.077591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01d90f-ea7d-4a58-899f-d4c313ba9e26 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:25.089722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:25.090956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:25.170742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.171357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.171630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.171893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.172530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.188061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.195484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1162/2344] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:25.226908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1183/2345] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:25.230735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:25.234301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:25.236039 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:25.247669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2c23d-136f-4dfc-9657-ac534406b998 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:25.254838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.262244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.262560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.262945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.263857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.265802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.280986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1163/2346] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:25.285337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:25.297206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f03c5-8f86-4f82-9a63-10d59bba043f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:25.304975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab6bf9-ffdf-4e2c-b305-de3cd63b6c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.306309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab6bf9-ffdf-4e2c-b305-de3cd63b6c2c admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.306309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab6bf9-ffdf-4e2c-b305-de3cd63b6c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.306309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab6bf9-ffdf-4e2c-b305-de3cd63b6c2c admin admin] PATH_INFO: `/v3/projects/3d5351c28cfa4491bfa37944aa187737/users/9b5f4bd410b544e29c4c569c225a93ed/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.345067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab6bf9-ffdf-4e2c-b305-de3cd63b6c2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5351c28cfa4491bfa37944aa187737, user_id=9b5f4bd410b544e29c4c569c225a93ed, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.345271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1ad7cb-2f37-4d93-b43d-ca42b4143f43 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:25.345385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab6bf9-ffdf-4e2c-b305-de3cd63b6c2c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.348484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.349391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.349899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.350189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.357882 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1164/2347] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:25 2022] PUT /identity/v3/projects/3d5351c28cfa4491bfa37944aa187737/users/9b5f4bd410b544e29c4c569c225a93ed/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:25.359920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5adb5-b95c-4add-ab62-9e834f01af86 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.360202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5adb5-b95c-4add-ab62-9e834f01af86 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.360471 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5adb5-b95c-4add-ab62-9e834f01af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.366521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.368037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.383487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0dd5adb5-b95c-4add-ab62-9e834f01af86 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:25.384747 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0dd5adb5-b95c-4add-ab62-9e834f01af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:25.392272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1184/2348] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:25.490432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1165/2349] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:25.501038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:25.503243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:25.519373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939cdf68-4075-4235-9d41-a3deba102320 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:25.606007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:25.606007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:25.606007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:25.606007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:25.620333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:25.621773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:25.634950 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1185/2350] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:48:27.420210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d571407-11dc-4308-a38c-1d01372fb3f4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:27.421384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d571407-11dc-4308-a38c-1d01372fb3f4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:27.422064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d571407-11dc-4308-a38c-1d01372fb3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:27.456859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-7d571407-11dc-4308-a38c-1d01372fb3f4 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:27.458895 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7d571407-11dc-4308-a38c-1d01372fb3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:27.465725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:27.562007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1166/2351] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:48:27 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:27.595138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:27.596918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:27.660931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:27.800186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2473de4b-fb05-4f73-b52c-6b12146351d4 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:27.840967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:27.843070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:27.843070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:27.843070 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:27.887391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:27.890422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:27.904122 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1186/2352] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:48:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:48:33.491755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:33.492374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:33.492374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:33.537585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] MFA Rules not processed for user `135e823ff7f64beab0689c3a84cc268f`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:33.538830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:33.724376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1167/2353] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:33.727704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:33.743454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06e8e9-7c52-4357-b6b2-01c76eacfb42 service swift] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:33.758933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:33.759129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:33.759382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:33.759565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:33.760185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:33.762296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:33.833666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1187/2354] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:33 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:33.837380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:33.857059 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8493788-4c0c-490d-b49a-89173f71a77c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:34.027851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:34.030057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:34.030513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:34.030953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:34.032356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:34.042405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.154170 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:34.156176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1168/2355] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:33 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:34.173881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca7260-e8b6-44f5-8127-80d47ef66958 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:34.206922 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:34.208189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:34.208189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:34.208189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:34.208657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:34.210328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.261486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1188/2356] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:48:34.266696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:34.282189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc42888-0480-4383-a5c0-07451c196810 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:34.294580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:34.295283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:34.295745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:34.296046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] PATH_INFO: `/v3/projects/5332eaf50eaf416daddbfc142e3917a8/users/b982664b971045f78b1a3813fb23080c/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:34.339358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5332eaf50eaf416daddbfc142e3917a8, user_id=b982664b971045f78b1a3813fb23080c, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:34.342161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.363612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1169/2357] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:34 2022] PUT /identity/v3/projects/5332eaf50eaf416daddbfc142e3917a8/users/b982664b971045f78b1a3813fb23080c/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:34.367618 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:34.384482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6325e82-2670-4298-b876-1e11ad3e6b94 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:34.498854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fd014-1b23-498b-b47c-7bd74640720e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:34.499802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd014-1b23-498b-b47c-7bd74640720e admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:34.500291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd014-1b23-498b-b47c-7bd74640720e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:34.500724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fd014-1b23-498b-b47c-7bd74640720e admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:34.501546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fd014-1b23-498b-b47c-7bd74640720e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:34.504588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fd014-1b23-498b-b47c-7bd74640720e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.518183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1189/2358] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:34 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:34.525658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:34.549007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f95d9-97ba-48e4-89d8-36321460c9b1 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:34.562774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:34.563918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:34.565708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:34.566193 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] PATH_INFO: `/v3/projects/5332eaf50eaf416daddbfc142e3917a8/users/b982664b971045f78b1a3813fb23080c/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:34.568834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5332eaf50eaf416daddbfc142e3917a8, user_id=b982664b971045f78b1a3813fb23080c, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:34.571947 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:34.581011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:34.600728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1170/2359] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:34 2022] PUT /identity/v3/projects/5332eaf50eaf416daddbfc142e3917a8/users/b982664b971045f78b1a3813fb23080c/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:40.012598 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:40.013152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:40.013152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:40.083481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] MFA Rules not processed for user `9b5f4bd410b544e29c4c569c225a93ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:40.084718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:40.306255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:40.316911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1190/2360] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:48:40.439364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b9e2e-2b37-4111-8190-087f4e0b1ea3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:40.446638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:40.447366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:40.447986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:40.448119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:40.448867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:40.452648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:40.504915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1171/2361] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:48:40 2022] POST /identity/v3/projects => generated 385 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:48:40.508857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:40.520907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:40.528866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f72f7-1ec6-4506-92d4-accbf56581ce None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:40.732328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:40.734530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:40.734821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:40.740144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:40.740933 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:40.744279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:40.816215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1191/2362] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:48:40 2022] POST /identity/v3/users => generated 299 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:48:40.820085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:40.839803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3a31d-6d38-47f2-9770-bf7d2d0b75ba admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:40.858991 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:40.860449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:40.860731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:40.861019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:40.861762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:40.864973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:40.891932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1172/2363] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:48:40.894901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:40.925183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8041e10-3354-4f03-a170-53cdb6df7c00 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:40.934436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:40.935125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:40.935433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:40.935682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] PATH_INFO: `/v3/projects/d5ee08a68485452a9a2f64746a5575c1/users/eb8afd5494044998a818568b8ed901b6/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:40.959822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ee08a68485452a9a2f64746a5575c1, user_id=eb8afd5494044998a818568b8ed901b6, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:40.962633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:40.996548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1192/2364] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:40 2022] PUT /identity/v3/projects/d5ee08a68485452a9a2f64746a5575c1/users/eb8afd5494044998a818568b8ed901b6/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:48:40.996651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:41.010089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be27f9-7054-4f9f-af9c-0256686a9b78 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:41.187086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c837a934-5e48-4926-a513-670c49c6d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:41.187887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837a934-5e48-4926-a513-670c49c6d33e admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:41.188178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837a934-5e48-4926-a513-670c49c6d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:41.188457 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c837a934-5e48-4926-a513-670c49c6d33e admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:41.189119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837a934-5e48-4926-a513-670c49c6d33e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:41.203446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c837a934-5e48-4926-a513-670c49c6d33e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:41.237597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1173/2365] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:48:41.238403 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:41.261339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d20a1-d98f-4004-8870-536382a17046 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:41.284879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:41.287966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:41.287966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:41.287966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] PATH_INFO: `/v3/projects/d5ee08a68485452a9a2f64746a5575c1/users/eb8afd5494044998a818568b8ed901b6/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:41.291062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5ee08a68485452a9a2f64746a5575c1, user_id=eb8afd5494044998a818568b8ed901b6, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:41.299337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:41.339440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1193/2366] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:48:41 2022] PUT /identity/v3/projects/d5ee08a68485452a9a2f64746a5575c1/users/eb8afd5494044998a818568b8ed901b6/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:48:46.637368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d71d8b-eea5-4dd9-b8b5-f69662ef0c3e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:46.637974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d71d8b-eea5-4dd9-b8b5-f69662ef0c3e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:46.637974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d71d8b-eea5-4dd9-b8b5-f69662ef0c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:46.709179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-03d71d8b-eea5-4dd9-b8b5-f69662ef0c3e None None] MFA Rules not processed for user `b982664b971045f78b1a3813fb23080c`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:46.717532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-03d71d8b-eea5-4dd9-b8b5-f69662ef0c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:46.823347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:46.907431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1174/2367] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:46.926321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:46.928436 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:47.176462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c994ea-deec-4857-8fae-17ebf9d760ac admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:47.186360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d0635e-8a27-440e-bf7a-98244f33967e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:47.187149 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0635e-8a27-440e-bf7a-98244f33967e service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:47.187551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0635e-8a27-440e-bf7a-98244f33967e service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:47.200254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d0635e-8a27-440e-bf7a-98244f33967e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:47.366907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:48:47.371584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0635e-8a27-440e-bf7a-98244f33967e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:47.373205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d0635e-8a27-440e-bf7a-98244f33967e service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:47.394368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1194/2368] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:51.727464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:51.728311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:51.728864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:51.758791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] MFA Rules not processed for user `135e823ff7f64beab0689c3a84cc268f`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:51.760716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:51.771515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1175/2369] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:52.603162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1b90a-0b4f-4f3a-90c0-755f90de75b6 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:52.603647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1b90a-0b4f-4f3a-90c0-755f90de75b6 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:52.604210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f1b90a-0b4f-4f3a-90c0-755f90de75b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:52.648204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-70f1b90a-0b4f-4f3a-90c0-755f90de75b6 None None] MFA Rules not processed for user `eb8afd5494044998a818568b8ed901b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:52.649702 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-70f1b90a-0b4f-4f3a-90c0-755f90de75b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:52.733087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1195/2370] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:48:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:52.745763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:52.752473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:52.820173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cb8042-dc25-455c-81d3-42505dd3313e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:52.824103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:52.824712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:52.824984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:52.825433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:52.894718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:52.896418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:52.908006 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1176/2371] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:48:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:48:53.043473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344446a4-01a2-4c35-abb5-8a3fb9a51d96 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:53.043473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344446a4-01a2-4c35-abb5-8a3fb9a51d96 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:53.044328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344446a4-01a2-4c35-abb5-8a3fb9a51d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:53.070896 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-344446a4-01a2-4c35-abb5-8a3fb9a51d96 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:53.072201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-344446a4-01a2-4c35-abb5-8a3fb9a51d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:53.133028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1196/2372] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:48:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:48:53.146588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:48:53.265325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b49116-8286-4248-bd4d-cda8624c84d7 service glance] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:48:53.276169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b4a6f7-6131-4917-bbd0-36a2a861e88e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:48:53.276989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4a6f7-6131-4917-bbd0-36a2a861e88e service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:53.277426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4a6f7-6131-4917-bbd0-36a2a861e88e service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:53.277997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b4a6f7-6131-4917-bbd0-36a2a861e88e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:53.293004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4a6f7-6131-4917-bbd0-36a2a861e88e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:48:53.294733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b4a6f7-6131-4917-bbd0-36a2a861e88e service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:48:53.306671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1177/2373] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:48:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:48:58.811921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28800e2a-050c-44e3-b6a0-d1a08b36d784 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:58.812944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28800e2a-050c-44e3-b6a0-d1a08b36d784 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:58.813358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28800e2a-050c-44e3-b6a0-d1a08b36d784 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:58.815960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1197/2374] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:48:58.832300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e18e-2f69-41fa-9acf-5488dbbf15f4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:48:58.833153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e18e-2f69-41fa-9acf-5488dbbf15f4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:48:58.833625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e18e-2f69-41fa-9acf-5488dbbf15f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:48:58.882971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-2576e18e-2f69-41fa-9acf-5488dbbf15f4 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:48:58.885245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2576e18e-2f69-41fa-9acf-5488dbbf15f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:48:58.909245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1178/2375] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:48:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:03.125152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=62 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:04.319450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28800e2a-050c-44e3-b6a0-d1a08b36d784 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:04.321137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-28800e2a-050c-44e3-b6a0-d1a08b36d784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:04.338331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28800e2a-050c-44e3-b6a0-d1a08b36d784 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:04.342042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606bfa9-d275-42f3-8294-1f6004045c6c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:04.342706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bfa9-d275-42f3-8294-1f6004045c6c service glance] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.342931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bfa9-d275-42f3-8294-1f6004045c6c service glance] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.343201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606bfa9-d275-42f3-8294-1f6004045c6c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.354065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606bfa9-d275-42f3-8294-1f6004045c6c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:04.355605 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4606bfa9-d275-42f3-8294-1f6004045c6c service glance] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:04.377404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1198/2376] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:04.402754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1623a-8372-4223-9c45-5d06778e2922 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.404678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1623a-8372-4223-9c45-5d06778e2922 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.404969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1623a-8372-4223-9c45-5d06778e2922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.426239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-80a1623a-8372-4223-9c45-5d06778e2922 None None] MFA Rules not processed for user `9b5f4bd410b544e29c4c569c225a93ed`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:04.427452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-80a1623a-8372-4223-9c45-5d06778e2922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:04.434837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1179/2377] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:49:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:04.460926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844256b1-b41c-443b-9f35-674d77b8aa16 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.461218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844256b1-b41c-443b-9f35-674d77b8aa16 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.461494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844256b1-b41c-443b-9f35-674d77b8aa16 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.462852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1199/2378] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:49:04.468330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd499-7b32-4e23-82c4-690158572958 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.468748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd499-7b32-4e23-82c4-690158572958 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.469115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd499-7b32-4e23-82c4-690158572958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.493211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-539fd499-7b32-4e23-82c4-690158572958 None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:04.494681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-539fd499-7b32-4e23-82c4-690158572958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:04.506562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:04.550602 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1180/2379] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:49:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:04.563303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844256b1-b41c-443b-9f35-674d77b8aa16 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:04.576891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844256b1-b41c-443b-9f35-674d77b8aa16 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:04.581113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5921b-aeea-46c7-b6e1-caba40efea43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:04.581909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5921b-aeea-46c7-b6e1-caba40efea43 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.582189 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5921b-aeea-46c7-b6e1-caba40efea43 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.582575 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5921b-aeea-46c7-b6e1-caba40efea43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.590154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5921b-aeea-46c7-b6e1-caba40efea43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:04.591578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad5921b-aeea-46c7-b6e1-caba40efea43 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:04.600696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1200/2380] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:04.693258 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c5078-0593-442c-abf2-a0dc924fe730 None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.693812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c5078-0593-442c-abf2-a0dc924fe730 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.694183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c5078-0593-442c-abf2-a0dc924fe730 None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.695644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1181/2381] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:49:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:49:04.701716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.702035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.702310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.724092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:04.725604 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:04.743975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1201/2382] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:49:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:04.754858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c5078-0593-442c-abf2-a0dc924fe730 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:04.766512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11c5078-0593-442c-abf2-a0dc924fe730 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:04.773240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df30fa6-e837-4078-a521-24923630250f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:04.773893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df30fa6-e837-4078-a521-24923630250f service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:04.774174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df30fa6-e837-4078-a521-24923630250f service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:04.774451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df30fa6-e837-4078-a521-24923630250f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:04.787777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df30fa6-e837-4078-a521-24923630250f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:04.789252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df30fa6-e837-4078-a521-24923630250f service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:04.806565 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1182/2383] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:10.104371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:10.106589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:10.185399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd696d83-0f47-469c-b10b-d7fc6bb24fe5 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:10.190864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:10.191482 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:10.191816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:10.192092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:10.193859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:10.206008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:10.207581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:10.217192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1202/2384] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:49:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:49:11.750897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43dce27-f487-4fbc-a221-c34e99b56c7d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:11.751430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43dce27-f487-4fbc-a221-c34e99b56c7d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:11.751771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43dce27-f487-4fbc-a221-c34e99b56c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:11.776680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d43dce27-f487-4fbc-a221-c34e99b56c7d None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:11.777963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d43dce27-f487-4fbc-a221-c34e99b56c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:11.788977 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1183/2385] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:49:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:11.803851 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:11.816847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc101de-60cf-4aca-9c8a-02e975ad46d8 service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:11.821882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:11.822546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:11.822825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:11.823811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:11.834586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:11.836244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:11.860477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1203/2386] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:49:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:49:21.769572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=25 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:21.842507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=53 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:29.280021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa548e8-95e0-41ca-89fe-59227b03ebdb None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:29.280479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa548e8-95e0-41ca-89fe-59227b03ebdb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:29.280681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa548e8-95e0-41ca-89fe-59227b03ebdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:29.302622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-baa548e8-95e0-41ca-89fe-59227b03ebdb None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:29.304498 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-baa548e8-95e0-41ca-89fe-59227b03ebdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:29.312171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1184/2387] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:49:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:49:29.326420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:49:29.337352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab2963-7251-4049-a217-a0dba727d54f service swift] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:49:29.342530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0988e-c9c2-4189-9859-01cbfd109a7d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:49:29.343212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0988e-c9c2-4189-9859-01cbfd109a7d service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:29.343581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0988e-c9c2-4189-9859-01cbfd109a7d service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:29.343946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0988e-c9c2-4189-9859-01cbfd109a7d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:29.352538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0988e-c9c2-4189-9859-01cbfd109a7d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:49:29.354241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0988e-c9c2-4189-9859-01cbfd109a7d service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:49:29.363401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1204/2388] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:49:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:49:39.296232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:39.359564 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:49:54.373497 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5481d-e1b9-41bb-bb62-a3ba2b542f19 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:49:54.374073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5481d-e1b9-41bb-bb62-a3ba2b542f19 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:49:54.374073 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f5481d-e1b9-41bb-bb62-a3ba2b542f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:49:54.393215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d0f5481d-e1b9-41bb-bb62-a3ba2b542f19 None None] MFA Rules not processed for user `8e9cefedd37f43e6bb64a270cde9f6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:49:54.394571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d0f5481d-e1b9-41bb-bb62-a3ba2b542f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:49:54.448940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1185/2389] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:49:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:00.371958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:00.372477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:00.372650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:00.393672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] MFA Rules not processed for user `c8b7cb87296a4ecea0b9ab44a0cbfd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:00.394786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:00.457572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1205/2390] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:50:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:00.663932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:00.664446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:00.664536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:00.685939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:00.687175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:00.697754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1186/2391] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 05:50:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:00.723341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:00.736623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a7eae-6706-4a33-91ea-cde71d91ac1f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:00.742342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:00.743590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:00.744033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:00.744464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:00.755363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:00.757589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:00.774637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1206/2392] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:50:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:50:04.082987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:04.093044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24974c-5939-4777-9f59-4874cfab0001 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:04.097472 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f69ce63-13b8-489e-8ec6-621ed98eeb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:04.098124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69ce63-13b8-489e-8ec6-621ed98eeb55 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:04.098406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69ce63-13b8-489e-8ec6-621ed98eeb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:04.098680 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f69ce63-13b8-489e-8ec6-621ed98eeb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:04.107792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69ce63-13b8-489e-8ec6-621ed98eeb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:04.109465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f69ce63-13b8-489e-8ec6-621ed98eeb55 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:04.118853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1187/2393] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:07.584281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:07.593589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85fa1a6-a6ce-429c-bbc9-7d32421e5009 service swift] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:07.597938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:07.598907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:07.599314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:07.599717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:07.611044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:07.612470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:07.623029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1207/2394] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:14.114699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=34 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:17.618174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=37 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:25.512698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:25.513224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:25.513224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:25.532669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] MFA Rules not processed for user `3d8467cc62d948038690af1fdf5bb96a`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:50:25.533830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:25.587542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1188/2395] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:50:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:50:35.580984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=25 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:50:50.009674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:50.018048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf110ee-dab9-437c-bea1-20965640a9aa service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:50.021544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:50.022152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:50.022426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:50.022694 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:50.024369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:50.037052 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:50.038512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:50.049404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1208/2396] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:50:51.235218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:50:51.244913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f84a47-ca63-4921-b364-00a64d220616 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:50:51.248513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:50:51.249113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:50:51.249397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:50:51.250504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:50:51.252000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:50:51.271130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:50:51.273116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:50:51.283845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1189/2397] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:50:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:51:00.044321 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:01.095862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:01.097304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:01.111634 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92f783-b080-4a4b-aeab-7390dadd8b4d service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:01.117110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:01.117735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:01.118015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:01.118287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:01.119940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:01.176946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:01.178686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:01.187917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1209/2398] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:01.278842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=6 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:03.409554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:03.411045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:03.475486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b495ec1-b300-45ea-b224-69ce4e60e3db service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:03.480095 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:03.480725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:03.481009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:03.481291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] PATH_INFO: `/v3/users/8e9cefedd37f43e6bb64a270cde9f6b6` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:03.482553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e9cefedd37f43e6bb64a270cde9f6b6)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:03.484743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:03.530529 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e9cefedd37f43e6bb64a270cde9f6b6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.531103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e9cefedd37f43e6bb64a270cde9f6b6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.538800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e9cefedd37f43e6bb64a270cde9f6b6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.539440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e9cefedd37f43e6bb64a270cde9f6b6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.548671 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4c3ab95b-3058-4375-976f-44e26e3db7a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e9cefedd37f43e6bb64a270cde9f6b6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.565696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1190/2399] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:51:03 2022] DELETE /identity/v3/users/8e9cefedd37f43e6bb64a270cde9f6b6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:51:03.744508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:03.755214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca10862-b03d-4536-971d-9c10585fb57b service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:03.826872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:03.827504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:03.827783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:03.828056 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] PATH_INFO: `/v3/projects/87f7a45fce204c9c9c15fae64b26b61e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:03.829763 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f7a45fce204c9c9c15fae64b26b61e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:03.831738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:03.886654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f7a45fce204c9c9c15fae64b26b61e', 'request_id': 'req-52084218-b33f-42ec-ab1e-30a574ae00b7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.895265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f7a45fce204c9c9c15fae64b26b61e', 'request_id': 'req-52084218-b33f-42ec-ab1e-30a574ae00b7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.904214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f7a45fce204c9c9c15fae64b26b61e', 'request_id': 'req-52084218-b33f-42ec-ab1e-30a574ae00b7'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.916462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] The token cache is being invalidate because project 87f7a45fce204c9c9c15fae64b26b61e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:51:03.916756 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.917459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:03.918901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1210/2400] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:51:03 2022] DELETE /identity/v3/projects/87f7a45fce204c9c9c15fae64b26b61e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:51:10.924156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:10.924666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:10.924777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:10.945179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] MFA Rules not processed for user `0b18352a23444324b27d17d2cee636fc`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:10.946503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:11.004463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1191/2401] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:51:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:51:11.511026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:11.512480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:11.544670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:11.572463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52084218-b33f-42ec-ab1e-30a574ae00b7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:11.575993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:11.576593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:11.576871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:11.577144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:11.579445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:11.588698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:11.590156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:11.598228 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1211/2402] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:11.900996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:11.902984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:11.934230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:11.977597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8416515a-772a-4519-8983-eb3328a7ca05 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:11.982477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:11.983074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:11.983607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:11.983880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:11.986114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:11.996967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:11.998588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:12.008942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1192/2403] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:16.563651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:16.565219 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:16.656024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b59206c-be7c-47e9-9858-7b1d96fbc9df service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:16.662420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:16.663048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:16.663329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:16.663600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:16.665397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:16.736026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:16.737645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:16.747727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1212/2404] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:22.004765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=21 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:22.694918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:22.696423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:22.755200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec0c6f-042e-4ebe-bce3-9d2b9e83de43 service placement] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:22.759078 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:22.759685 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:22.759983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:22.760422 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:22.762201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:22.822072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:22.823535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee8298f-f13e-4d2a-902a-bffb1edf7455 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:22.833008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1193/2405] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:22 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:26.743832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=69 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:31.458698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:31.470736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c21479-4575-4c2c-a129-b4bc7828cdb5 service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:31.476194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:31.476793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:31.477068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:31.477406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:31.479498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:31.493401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:31.495132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:31.507257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1213/2406] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:32.828970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=48 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:38.351001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:38.351667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:38.351667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:38.371048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] MFA Rules not processed for user `c8b7cb87296a4ecea0b9ab44a0cbfd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:38.372261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:38.379912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1194/2407] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:51:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:51:41.502467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:41.974286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:41.975703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:41.988945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce498deb-a8aa-4423-91a5-3c8a70988546 service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:41.993777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:41.994365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:41.994647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:41.994920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:41.997121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:42.008281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:42.009760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:42.021944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1214/2408] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:48.365661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:48.589191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:48.598774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d6216-80f0-401d-893b-e2cff30bee9b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:48.602292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:48.602879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:48.603155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:48.603419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:48.605086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:48.667003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:48.668434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:48.678670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1195/2409] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:51.836105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:51.857509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530670b0-7f1b-4901-94a9-18e2cd223247 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:51.873621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:51.874243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:51.875625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:51.875914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:51.880421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:51.912295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:51.915419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:51.940253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1215/2410] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:52.265826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:52.268271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:52.293890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac3a9bc-38aa-4856-a222-35828b19c2d4 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:52.302467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:52.303470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:52.304645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:52.305090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:52.307294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:52.330713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:52.332358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:52.363363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1196/2411] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:54.026038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.027558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:54.042812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1f91da-6cda-4d12-8f86-f88acfbc91bb service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.047931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.048557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.048835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.049106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] PATH_INFO: `/v3/users/3d8467cc62d948038690af1fdf5bb96a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.050252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d8467cc62d948038690af1fdf5bb96a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.052139 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.101002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d8467cc62d948038690af1fdf5bb96a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.101667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d8467cc62d948038690af1fdf5bb96a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.110643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d8467cc62d948038690af1fdf5bb96a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.111270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d8467cc62d948038690af1fdf5bb96a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.123229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-cc02ad4d-58eb-49c7-861d-261a15b30e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d8467cc62d948038690af1fdf5bb96a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.135556 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1216/2412] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:51:54 2022] DELETE /identity/v3/users/3d8467cc62d948038690af1fdf5bb96a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:51:54.307433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.323729 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c829e4-8077-4ef6-9068-32e53d7ed22f service nova] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.382005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.382638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.382921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.383197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] PATH_INFO: `/v3/projects/3726a58288d9455b97e041a2e92f8850` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.384317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3726a58288d9455b97e041a2e92f8850)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.386287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.433319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3726a58288d9455b97e041a2e92f8850', 'request_id': 'req-3dedb76f-e867-4650-a8bf-bba0f16759c1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.438930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3726a58288d9455b97e041a2e92f8850', 'request_id': 'req-3dedb76f-e867-4650-a8bf-bba0f16759c1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.445025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3726a58288d9455b97e041a2e92f8850', 'request_id': 'req-3dedb76f-e867-4650-a8bf-bba0f16759c1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.457687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] The token cache is being invalidate because project 3726a58288d9455b97e041a2e92f8850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:51:54.457979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.458601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3dedb76f-e867-4650-a8bf-bba0f16759c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:51:54.460074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1197/2413] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:51:54 2022] DELETE /identity/v3/projects/3726a58288d9455b97e041a2e92f8850 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:51:54.465033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.465333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.465663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.486063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:54.487197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:54.543454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1217/2414] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:51:54.552469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.552765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.553043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.572336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:54.573595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:54.581750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1198/2415] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:51:54.585998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.596152 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015da9d5-87d0-44cf-b5b1-38ff0f66174a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.600629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.601221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.601494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.601837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.602511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:51:54.602804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.604375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.616728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1218/2416] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:51:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:51:54.621086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.626767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:54.630385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb4e42-14ee-42b2-8cd6-cd00f3d3493e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.636447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.637212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.637690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.638096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.638851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.640834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.668512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1199/2417] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:51:54 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:51:54.673205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.683515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b19f01-ffe3-4290-8c18-010cec69ea9d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.714363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:51:54.751792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.752793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.753103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.753385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.754201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.756419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.783204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1219/2418] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:51:54 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:51:54.792511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.803695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a400a374-81f8-4d3d-9d22-379102a0f752 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.813837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.814508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.814910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.815186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.822155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.824300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.838839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1200/2419] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:51:54.843594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.856537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c23b4c-0fe9-4e51-902e-edc920e4ae10 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.863600 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034e610-82d3-4eb3-9150-31890ccf1370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.864552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034e610-82d3-4eb3-9150-31890ccf1370 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.864867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034e610-82d3-4eb3-9150-31890ccf1370 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.865140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034e610-82d3-4eb3-9150-31890ccf1370 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.865816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034e610-82d3-4eb3-9150-31890ccf1370 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.867691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034e610-82d3-4eb3-9150-31890ccf1370 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.881599 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1220/2420] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:51:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:51:54.883170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:54.894683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646253d-df75-4862-82e1-5bf23669ad1c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:54.899558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:54.900309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.900715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.901058 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] PATH_INFO: `/v3/projects/b93f8c24b94841168ae2212ae15ecde9/users/962fc6a6c7c04d6f93109c420c5aa6cd/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.917348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b93f8c24b94841168ae2212ae15ecde9, user_id=962fc6a6c7c04d6f93109c420c5aa6cd, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:54.920268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:54.936914 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1201/2421] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:51:54 2022] PUT /identity/v3/projects/b93f8c24b94841168ae2212ae15ecde9/users/962fc6a6c7c04d6f93109c420c5aa6cd/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:51:54.942081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2df58-ecc4-4a54-98e1-f943a259e6fd None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:54.942380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2df58-ecc4-4a54-98e1-f943a259e6fd None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:54.942669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2df58-ecc4-4a54-98e1-f943a259e6fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:54.962135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-80f2df58-ecc4-4a54-98e1-f943a259e6fd None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:54.963342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-80f2df58-ecc4-4a54-98e1-f943a259e6fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:55.024414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1221/2422] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:51:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:51:55.034550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:51:55.036343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:51:55.093916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab20f0-cb5f-413c-9dcd-37c0185f281c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:51:55.097530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:51:55.098180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:55.098447 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:55.098718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:55.107616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:51:55.109321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:51:55.118711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1202/2423] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:51:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:51:59.917839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1acd-53f3-494a-a4c2-b406857f0e4c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:51:59.918381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1acd-53f3-494a-a4c2-b406857f0e4c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:51:59.918512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce1acd-53f3-494a-a4c2-b406857f0e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:51:59.959091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a4ce1acd-53f3-494a-a4c2-b406857f0e4c None None] MFA Rules not processed for user `962fc6a6c7c04d6f93109c420c5aa6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:51:59.960461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a4ce1acd-53f3-494a-a4c2-b406857f0e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:00.040360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1222/2424] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:51:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:00.055410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:00.056965 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:00.168938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbe50e-854c-4b51-9bca-e51ad4fb77e1 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:00.169264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbe50e-854c-4b51-9bca-e51ad4fb77e1 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:00.169542 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbe50e-854c-4b51-9bca-e51ad4fb77e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:00.185307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8c358-45b6-4bed-b295-88de26b00ef1 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:00.190383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-46cbe50e-854c-4b51-9bca-e51ad4fb77e1 None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:00.191608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-46cbe50e-854c-4b51-9bca-e51ad4fb77e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:00.192444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:00.193123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:00.193396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:00.193738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:00.202732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:00.204330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:00.213235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1203/2425] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:52:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:52:00.232069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:00.257927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1223/2426] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 05:52:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:52:00.313826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:00.315221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:00.363709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:00.395912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87e91d-cbf5-464a-9498-635b23d332ab service nova] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:00.400932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:00.401664 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:00.402041 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:00.402349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:00.415830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:00.417440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:00.428723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1204/2427] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:52:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 05:52:02.070190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:02.070695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:02.070790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:02.102079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] MFA Rules not processed for user `962fc6a6c7c04d6f93109c420c5aa6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:02.105925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:02.118237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1224/2428] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:52:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:02.752435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:02.754480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:02.768891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282f366-7644-4476-a840-77f76fab8d08 service swift] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:02.773104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:02.774056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:02.774339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:02.774610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:02.780859 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:02.792605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:02.794433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:02.803904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1205/2429] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:52:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:52:08.152778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:08.158545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:08.174099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd480814-e33f-4e4b-9842-33ea1154616a None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:08.179273 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:08.180255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:08.180642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:08.181041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] PATH_INFO: `/v3/users/9b5f4bd410b544e29c4c569c225a93ed` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:08.182398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b5f4bd410b544e29c4c569c225a93ed)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:08.184576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:08.244956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b5f4bd410b544e29c4c569c225a93ed'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.245761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b5f4bd410b544e29c4c569c225a93ed'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.262733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b5f4bd410b544e29c4c569c225a93ed'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.263973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b5f4bd410b544e29c4c569c225a93ed'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.294327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b5f4bd410b544e29c4c569c225a93ed'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.304397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1225/2430] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:52:08 2022] DELETE /identity/v3/users/9b5f4bd410b544e29c4c569c225a93ed => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:52:08.478372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:08.492833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5c3099-f0c4-4419-a968-d37d50e586dc service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:08.553860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:08.554404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:08.554709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:08.554980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] PATH_INFO: `/v3/projects/3d5351c28cfa4491bfa37944aa187737` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:08.556082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d5351c28cfa4491bfa37944aa187737)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:08.558011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:08.602955 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d5351c28cfa4491bfa37944aa187737', 'request_id': 'req-0f8718af-d332-49b2-bbd4-ba83b5283c50'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.609780 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d5351c28cfa4491bfa37944aa187737', 'request_id': 'req-0f8718af-d332-49b2-bbd4-ba83b5283c50'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.615834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d5351c28cfa4491bfa37944aa187737', 'request_id': 'req-0f8718af-d332-49b2-bbd4-ba83b5283c50'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.629018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] The token cache is being invalidate because project 3d5351c28cfa4491bfa37944aa187737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:52:08.629302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.630000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.631461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1206/2431] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:52:08 2022] DELETE /identity/v3/projects/3d5351c28cfa4491bfa37944aa187737 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:52:08.635557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:08.636959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:08.650037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac0de2-3d47-4528-8ee8-cb6ad1772e76 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:08.707676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:08.708310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:08.708589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:08.708863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] PATH_INFO: `/v3/users/eb8afd5494044998a818568b8ed901b6` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:08.710027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb8afd5494044998a818568b8ed901b6)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:08.711943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:08.759909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb8afd5494044998a818568b8ed901b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.760553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb8afd5494044998a818568b8ed901b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.767805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb8afd5494044998a818568b8ed901b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.768429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb8afd5494044998a818568b8ed901b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.769538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:08.783390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3add5bac-f5c2-4b6c-a94d-f542e97fac4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb8afd5494044998a818568b8ed901b6'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:08.795254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1226/2432] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:52:08 2022] DELETE /identity/v3/users/eb8afd5494044998a818568b8ed901b6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:52:08.985940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:08.988293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:09.008143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8718af-d332-49b2-bbd4-ba83b5283c50 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:09.058523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:09.078417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:09.078901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:09.079177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:09.079443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] PATH_INFO: `/v3/projects/d5ee08a68485452a9a2f64746a5575c1` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:09.080648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5ee08a68485452a9a2f64746a5575c1)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:09.082618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:09.129538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5ee08a68485452a9a2f64746a5575c1', 'request_id': 'req-7df74e40-cc13-4a62-80f6-45db1bfb34f1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:09.135178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5ee08a68485452a9a2f64746a5575c1', 'request_id': 'req-7df74e40-cc13-4a62-80f6-45db1bfb34f1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:09.142061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5ee08a68485452a9a2f64746a5575c1', 'request_id': 'req-7df74e40-cc13-4a62-80f6-45db1bfb34f1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:09.154348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] The token cache is being invalidate because project d5ee08a68485452a9a2f64746a5575c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:52:09.154634 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:09.155888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:52:09.157363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1207/2433] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:52:08 2022] DELETE /identity/v3/projects/d5ee08a68485452a9a2f64746a5575c1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:52:18.791788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=11,keystone:DELETE=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:19.151317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=32,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:28.200704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73940c-c2b9-473f-8721-d260c377cb6a None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:28.201257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73940c-c2b9-473f-8721-d260c377cb6a None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:28.201257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73940c-c2b9-473f-8721-d260c377cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:28.220897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9f73940c-c2b9-473f-8721-d260c377cb6a None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:28.222215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9f73940c-c2b9-473f-8721-d260c377cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:28.282637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1227/2434] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:52:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:28.295017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:28.296402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:28.371235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df74e40-cc13-4a62-80f6-45db1bfb34f1 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:28.378245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6bc881-0c13-400c-9d94-cb73dc6cf6fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:28.378860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc881-0c13-400c-9d94-cb73dc6cf6fc service swift] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:28.379134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc881-0c13-400c-9d94-cb73dc6cf6fc service swift] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:28.379404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6bc881-0c13-400c-9d94-cb73dc6cf6fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:28.389444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bc881-0c13-400c-9d94-cb73dc6cf6fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:28.391153 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6bc881-0c13-400c-9d94-cb73dc6cf6fc service swift] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:28.400089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1208/2435] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:52:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:52:38.275997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=25 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:38.396346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=32 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:52:44.742766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeeff5-b525-4bd3-a0e6-facf35487e52 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:44.743285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeeff5-b525-4bd3-a0e6-facf35487e52 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:44.743382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baeeff5-b525-4bd3-a0e6-facf35487e52 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:44.744882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1228/2436] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:52:44.762957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744952d-9250-4b5e-88de-12379af1e217 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:44.763476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744952d-9250-4b5e-88de-12379af1e217 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:44.763580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744952d-9250-4b5e-88de-12379af1e217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:44.792352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3744952d-9250-4b5e-88de-12379af1e217 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:44.793730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3744952d-9250-4b5e-88de-12379af1e217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:44.868065 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1209/2437] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:52:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:52:44.880218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeeff5-b525-4bd3-a0e6-facf35487e52 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:44.882001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5baeeff5-b525-4bd3-a0e6-facf35487e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:44.943852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baeeff5-b525-4bd3-a0e6-facf35487e52 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:44.947423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2510c1bd-6a00-4264-83f9-11ff70a3c412 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:44.948035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510c1bd-6a00-4264-83f9-11ff70a3c412 service glance] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:44.948317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510c1bd-6a00-4264-83f9-11ff70a3c412 service glance] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:44.948587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2510c1bd-6a00-4264-83f9-11ff70a3c412 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:44.959351 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510c1bd-6a00-4264-83f9-11ff70a3c412 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:44.960770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2510c1bd-6a00-4264-83f9-11ff70a3c412 service glance] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:44.971463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1229/2438] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:52:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:52:45.044237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96462a12-19e1-4e86-bf00-6c19d6d39bfc None None] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:45.044518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96462a12-19e1-4e86-bf00-6c19d6d39bfc None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:45.044794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96462a12-19e1-4e86-bf00-6c19d6d39bfc None None] PATH_INFO: `/` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:45.046167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1210/2439] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:52:45.051414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5963-4956-4a07-a455-5022c24691bb None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:45.051761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5963-4956-4a07-a455-5022c24691bb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:45.052053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f5963-4956-4a07-a455-5022c24691bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:45.071798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-216f5963-4956-4a07-a455-5022c24691bb None None] MFA Rules not processed for user `20154db5efc6482bbae149b81fb06cd2`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:45.072981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-216f5963-4956-4a07-a455-5022c24691bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:45.127496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1230/2440] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:52:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:45.139359 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96462a12-19e1-4e86-bf00-6c19d6d39bfc None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:45.141092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-96462a12-19e1-4e86-bf00-6c19d6d39bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:45.156179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96462a12-19e1-4e86-bf00-6c19d6d39bfc None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:45.160366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aa6439-4c1e-490e-8431-bf141b052c82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:45.160969 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6439-4c1e-490e-8431-bf141b052c82 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:45.161279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6439-4c1e-490e-8431-bf141b052c82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:45.161557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aa6439-4c1e-490e-8431-bf141b052c82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:45.170517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa6439-4c1e-490e-8431-bf141b052c82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:45.173094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aa6439-4c1e-490e-8431-bf141b052c82 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:45.182443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1211/2441] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:52:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:52:45.271184 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6be40-a3ff-41e1-b99b-b40ee1dc9e82 None None] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:45.271848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6be40-a3ff-41e1-b99b-b40ee1dc9e82 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:45.272521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6be40-a3ff-41e1-b99b-b40ee1dc9e82 None None] PATH_INFO: `/` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:45.274091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1231/2442] 10.222.0.73 () {54 vars in 921 bytes} [Wed Aug 24 05:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 05:52:45.279961 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b4824-c690-4e8d-893c-a8870f0af4b1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:45.280242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b4824-c690-4e8d-893c-a8870f0af4b1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:45.280513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b4824-c690-4e8d-893c-a8870f0af4b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:45.300800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ae9b4824-c690-4e8d-893c-a8870f0af4b1 None None] MFA Rules not processed for user `4e926d4a438e42ecaeab44f711e0b3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:45.302030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ae9b4824-c690-4e8d-893c-a8870f0af4b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:45.311187 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1212/2443] 10.222.0.73 () {58 vars in 1018 bytes} [Wed Aug 24 05:52:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:45.321786 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6be40-a3ff-41e1-b99b-b40ee1dc9e82 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:45.323478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ee6be40-a3ff-41e1-b99b-b40ee1dc9e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:45.342373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6be40-a3ff-41e1-b99b-b40ee1dc9e82 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:45.346594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:45.347200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:45.347478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:45.347752 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:45.358590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:45.360680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:45.371177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1232/2444] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:52:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:52:50.232837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab53e7-0d13-4468-9a2d-dfc224aac82b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:50.233818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab53e7-0d13-4468-9a2d-dfc224aac82b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:50.234213 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ab53e7-0d13-4468-9a2d-dfc224aac82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:50.264092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-64ab53e7-0d13-4468-9a2d-dfc224aac82b None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:52:50.265429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-64ab53e7-0d13-4468-9a2d-dfc224aac82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:52:50.273549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1213/2445] 10.222.0.73 () {58 vars in 1009 bytes} [Wed Aug 24 05:52:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:52:50.284911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:52:50.300232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378df06-97fb-448f-a5b8-b01c7cefb965 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:52:50.306110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:52:50.306784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:52:50.307073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:52:50.307343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:52:50.316736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:52:50.318473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:52:50.327959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1233/2446] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:52:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:53:00.252746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=43 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:00.324562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=61 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:05.892673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:05.893123 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:05.893229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:05.913612 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] MFA Rules not processed for user `cca6991f4cff4e8b94c3d8b727fb6633`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:05.914850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:05.922136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1214/2447] 10.222.0.73 () {58 vars in 967 bytes} [Wed Aug 24 05:53:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:05.943514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:05.954366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224041a-b0a1-474e-8a77-44b35c04e515 service swift] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:05.959504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cdca2e846d9340a5ab6ee8356115a6c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:05.960100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:05.960377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:05.960645 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:05.969339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:05.970945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:05.978843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1234/2448] 10.222.0.73 () {60 vars in 1384 bytes} [Wed Aug 24 05:53:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 05:53:07.466621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:07.476322 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f349a-9952-4cfc-99b6-9d66131aac4c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:07.480110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:07.480590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:07.480885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:07.481180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:07.483068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:07.546328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:07.548174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:07.558084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1215/2449] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:10.789958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:10.792955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:10.809905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab42d1-3d58-4aab-9cce-def0aecf8c87 service swift] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:10.815145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:10.815793 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:10.816145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:10.816308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] PATH_INFO: `/v3/users/c8b7cb87296a4ecea0b9ab44a0cbfd2c` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:10.817648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b7cb87296a4ecea0b9ab44a0cbfd2c)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:10.819557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:10.881257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b7cb87296a4ecea0b9ab44a0cbfd2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:10.882037 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b7cb87296a4ecea0b9ab44a0cbfd2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:10.894111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b7cb87296a4ecea0b9ab44a0cbfd2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:10.896665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b7cb87296a4ecea0b9ab44a0cbfd2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:10.918432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-40f874b8-af0e-4a53-8b65-3b6ad142b822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b7cb87296a4ecea0b9ab44a0cbfd2c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:10.932294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1235/2450] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:10 2022] DELETE /identity/v3/users/c8b7cb87296a4ecea0b9ab44a0cbfd2c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:11.137222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.148017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328e6c-f791-4fcd-a757-67f59335f0a9 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.207138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.207776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.208056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.208327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] PATH_INFO: `/v3/projects/57dfc5e65dc1412ab199fff503acbfc3` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.209444 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] RBAC: Authorizing `identity:delete_project(project_id=57dfc5e65dc1412ab199fff503acbfc3)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.211407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.262347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57dfc5e65dc1412ab199fff503acbfc3', 'request_id': 'req-ae1c09b7-d31f-4e95-9634-9db67529221e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:11.268414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57dfc5e65dc1412ab199fff503acbfc3', 'request_id': 'req-ae1c09b7-d31f-4e95-9634-9db67529221e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:11.274667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57dfc5e65dc1412ab199fff503acbfc3', 'request_id': 'req-ae1c09b7-d31f-4e95-9634-9db67529221e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:11.287615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] The token cache is being invalidate because project 57dfc5e65dc1412ab199fff503acbfc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:11.287904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:11.288527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ae1c09b7-d31f-4e95-9634-9db67529221e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:11.290040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1216/2451] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:11 2022] DELETE /identity/v3/projects/57dfc5e65dc1412ab199fff503acbfc3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:11.294908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.295254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.295536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.314656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:11.315854 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:11.374267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1236/2452] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:11.382135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.382416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.382687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.403048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:11.404243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:11.412987 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1217/2453] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:11.417232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.426862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23d533-ab6b-4df4-9178-908dc1b74279 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.431576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.432178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.432456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.432728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.433369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:11.433704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.435397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.448146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1237/2454] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:11.452382 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.462056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a12fc-23e5-4d14-b4df-718dd54c6c84 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.466633 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.467228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.467493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.467759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.468375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.471099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.480376 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:11.498630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1218/2455] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:11 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:11.503243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.513104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55789674-a437-4c29-a813-33760d5d2afc admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.557607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:11.572094 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.572697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.572978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.573246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.573898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.575787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.596993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1238/2456] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:11 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:11.601138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.611799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeeddfa-366e-42fa-81cf-78d88f781bee admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.616665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.617260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.617557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.617843 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.618480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.620098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.630507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1219/2457] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:11.634386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.644986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fef72d-9a62-441b-8536-0d13add94253 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.649840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb76072f-a728-4b21-8f98-71cdb20f0d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.650438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76072f-a728-4b21-8f98-71cdb20f0d67 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.650715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76072f-a728-4b21-8f98-71cdb20f0d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.651048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb76072f-a728-4b21-8f98-71cdb20f0d67 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.651871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76072f-a728-4b21-8f98-71cdb20f0d67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.653707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb76072f-a728-4b21-8f98-71cdb20f0d67 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.664644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1239/2458] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:53:11.668722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.678598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8867117c-34db-47de-bb56-4480293edda0 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.683226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.683867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.684143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.684419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] PATH_INFO: `/v3/projects/58962a2f968c4e28909cab3a1856e1b6/users/e5217041950a48f78f40259782a298d3/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.699477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58962a2f968c4e28909cab3a1856e1b6, user_id=e5217041950a48f78f40259782a298d3, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.702183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.716053 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1220/2459] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:11 2022] PUT /identity/v3/projects/58962a2f968c4e28909cab3a1856e1b6/users/e5217041950a48f78f40259782a298d3/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:11.726299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.726589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.726864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.754603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] MFA Rules not processed for user `e5217041950a48f78f40259782a298d3`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:11.755931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:11.819049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1240/2460] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:11.831050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:11.832765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:11.904531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f93af-7806-410b-8f0d-1e5fba4387f6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:11.910010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bbb704-4b54-482d-9e75-30a506e580d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:11.910607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb704-4b54-482d-9e75-30a506e580d4 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:11.910884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb704-4b54-482d-9e75-30a506e580d4 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:11.911156 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bbb704-4b54-482d-9e75-30a506e580d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:11.918739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbb704-4b54-482d-9e75-30a506e580d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:11.920300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bbb704-4b54-482d-9e75-30a506e580d4 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:11.929358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1221/2461] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:12.138357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:12.202701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796fe00-5c89-4745-8568-12dfdf3b4e26 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:12.207244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:12.207871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:12.208156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:12.208423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] PATH_INFO: `/v3/users/e5217041950a48f78f40259782a298d3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:12.209538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5217041950a48f78f40259782a298d3)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:12.211500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:12.263981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5217041950a48f78f40259782a298d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.264641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5217041950a48f78f40259782a298d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.274013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5217041950a48f78f40259782a298d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.274657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5217041950a48f78f40259782a298d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.276776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:12.290003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5217041950a48f78f40259782a298d3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.304244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1241/2462] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:12 2022] DELETE /identity/v3/users/e5217041950a48f78f40259782a298d3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:12.308687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:12.309216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:12.309216 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:12.327949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:12.329083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:12.388101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1222/2463] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:12.397816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:12.399202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:12.455778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05baf7d4-0fca-4cd6-8861-e5aeb8efe54f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:12.459378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856fb7fc-a4b9-4b4f-8dc7-f5fce2a81c9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:12.459993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fb7fc-a4b9-4b4f-8dc7-f5fce2a81c9c service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:12.460261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fb7fc-a4b9-4b4f-8dc7-f5fce2a81c9c service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:12.460527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856fb7fc-a4b9-4b4f-8dc7-f5fce2a81c9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:12.469969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fb7fc-a4b9-4b4f-8dc7-f5fce2a81c9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:12.471388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856fb7fc-a4b9-4b4f-8dc7-f5fce2a81c9c service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:12.480596 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1242/2464] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:12.753076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:12.754513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:12.766807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e8a57-31a0-420c-915d-7436877ac9f7 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:12.771631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:12.772261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:12.777111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:12.777419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] PATH_INFO: `/v3/projects/58962a2f968c4e28909cab3a1856e1b6` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:12.778676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=58962a2f968c4e28909cab3a1856e1b6)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:12.780652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:12.819520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:12.831614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58962a2f968c4e28909cab3a1856e1b6', 'request_id': 'req-b3493ca9-9b85-479e-83c1-d336f95b6bfe'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.836920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58962a2f968c4e28909cab3a1856e1b6', 'request_id': 'req-b3493ca9-9b85-479e-83c1-d336f95b6bfe'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.843003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58962a2f968c4e28909cab3a1856e1b6', 'request_id': 'req-b3493ca9-9b85-479e-83c1-d336f95b6bfe'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.855954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] The token cache is being invalidate because project 58962a2f968c4e28909cab3a1856e1b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:12.856244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.856889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b3493ca9-9b85-479e-83c1-d336f95b6bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:12.858692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1223/2465] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:12 2022] DELETE /identity/v3/projects/58962a2f968c4e28909cab3a1856e1b6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:12.863455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:12.863901 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:12.863993 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:12.883295 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:12.884433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:12.945041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1243/2466] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:12.953383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:12.953705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:12.954003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:12.973934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:12.975141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:12.983071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1224/2467] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:12.987404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:12.997022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44320418-3325-454d-8c1a-41f1b3c90ec4 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.002112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.002711 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.002995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.003264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.003903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:13.004204 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.005834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.017700 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1244/2468] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:13.021948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:13.031256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a945e-82f0-470e-9826-b6205235367d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.036054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.036648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.036927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.037197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.037853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.039745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.068139 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1225/2469] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:13 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:13.072342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:13.082804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4397dc-dcd9-43bd-bf94-9b579e957411 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.137869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.138387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.138663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.138931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.139552 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.142112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.151821 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:13.165861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1245/2470] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:13 2022] POST /identity/v3/users => generated 303 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:53:13.170021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:13.180248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feed093-e421-4666-bf4f-dd28e93a7498 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.185199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.185866 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.186165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.186431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.187580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.189167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.199134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1226/2471] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:13.203650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:13.213522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a8459c-8759-4602-8807-b21fe78377a0 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.218491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738edb0-e42e-47c2-88a6-16eb2c40a210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.219111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738edb0-e42e-47c2-88a6-16eb2c40a210 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.219391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738edb0-e42e-47c2-88a6-16eb2c40a210 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.219665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738edb0-e42e-47c2-88a6-16eb2c40a210 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.220312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738edb0-e42e-47c2-88a6-16eb2c40a210 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.222020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738edb0-e42e-47c2-88a6-16eb2c40a210 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.233050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1246/2472] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:13.237610 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:13.247703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9262d-0ede-4627-939a-3a29f7696bea admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.253086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.253767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.254047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.254320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] PATH_INFO: `/v3/projects/16f0c974e77341dea999141ee460c0c2/users/ea02c1c3fb414736a97d8896d3d6e624/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.272503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16f0c974e77341dea999141ee460c0c2, user_id=ea02c1c3fb414736a97d8896d3d6e624, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.275185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.289235 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1227/2473] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:13 2022] PUT /identity/v3/projects/16f0c974e77341dea999141ee460c0c2/users/ea02c1c3fb414736a97d8896d3d6e624/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:13.294281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d359c30-6485-432f-8c07-d098902fc9b0 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.294567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d359c30-6485-432f-8c07-d098902fc9b0 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.294839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d359c30-6485-432f-8c07-d098902fc9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.313612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3d359c30-6485-432f-8c07-d098902fc9b0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:13.314823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3d359c30-6485-432f-8c07-d098902fc9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:13.375594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1247/2474] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:13.400150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:13.402117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:13.460776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09820b8b-d80e-4e15-8521-a16894f0870b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:13.464433 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:13.465957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:13.466098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:13.466328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:13.476124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:13.477641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:13.488344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1228/2475] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:14.098825 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.099349 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.099797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.149409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] MFA Rules not processed for user `ea02c1c3fb414736a97d8896d3d6e624`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:14.151647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:14.226589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1248/2476] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:14 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:14.231339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:14.242323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec788ca1-db09-4396-a16d-07698aa3e782 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:14.247389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:14.248003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.248280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.248548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.249350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:14.252650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:14.286292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1229/2477] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:14 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:14.287589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:14.299507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebdb9c1-cfcb-49d8-81cd-9b6b08b9323e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:14.357873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:14.358735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.359510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.359777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.360391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:14.362319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:14.389262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1249/2478] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:14 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:14.390651 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:14.401490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e8857-6241-43cf-81ae-a8546c0b599b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:14.407890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:14.407890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.407890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.407890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.408348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:14.409689 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:14.420404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1230/2479] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:53:14.425158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:14.431049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:14.438134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d25da6-80a8-4364-8b5c-9a9d09cf4f0f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:14.443125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:14.443784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.444068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.444341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] PATH_INFO: `/v3/projects/76004352e42d44baa87d32d0bcfb598b/users/be474eb968de4dd183b382b680b0ee7e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.459265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] RBAC: Authorizing `identity:create_grant(project_id=76004352e42d44baa87d32d0bcfb598b, user_id=be474eb968de4dd183b382b680b0ee7e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:14.461932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:14.475964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1250/2480] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:14 2022] PUT /identity/v3/projects/76004352e42d44baa87d32d0bcfb598b/users/be474eb968de4dd183b382b680b0ee7e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:14.479959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:14.490972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca010d4-94c0-4828-8d99-44c652faae3d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:14.516411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:14.549894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc86b2-d39e-4b71-b861-09903ee93b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:14.550532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc86b2-d39e-4b71-b861-09903ee93b7c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.550995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc86b2-d39e-4b71-b861-09903ee93b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.551276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc86b2-d39e-4b71-b861-09903ee93b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.551924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc86b2-d39e-4b71-b861-09903ee93b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:14.553530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc86b2-d39e-4b71-b861-09903ee93b7c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:14.564133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1231/2481] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:14.568356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:14.579650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d4d46-57c7-4898-a578-5ee63cbc432f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:14.585673 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:14.586368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:14.586655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:14.586928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] PATH_INFO: `/v3/projects/76004352e42d44baa87d32d0bcfb598b/users/be474eb968de4dd183b382b680b0ee7e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:14.588874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76004352e42d44baa87d32d0bcfb598b, user_id=be474eb968de4dd183b382b680b0ee7e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:14.592220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:14.603670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1251/2482] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:14 2022] PUT /identity/v3/projects/76004352e42d44baa87d32d0bcfb598b/users/be474eb968de4dd183b382b680b0ee7e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:15.225751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:15.226137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:15.226418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:15.261131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] MFA Rules not processed for user `be474eb968de4dd183b382b680b0ee7e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:15.262517 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:15.336902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1232/2483] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:15 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:15.842761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:15.844660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:15.922843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16063116-c360-435b-8b15-330c30a05d98 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:15.927140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:15.927765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:15.928045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:15.928322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:15.930236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:15.943098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:15.944803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:15.954881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1252/2484] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:16.429648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:16.491531 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1839a-86c2-4230-9e15-d2c0dd1c3875 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:16.496212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:16.497077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:16.497356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:16.497879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] PATH_INFO: `/v3/users/ea02c1c3fb414736a97d8896d3d6e624` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:16.499201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea02c1c3fb414736a97d8896d3d6e624)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:16.501134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:16.546467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea02c1c3fb414736a97d8896d3d6e624'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.547124 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea02c1c3fb414736a97d8896d3d6e624'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.551598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:16.555332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea02c1c3fb414736a97d8896d3d6e624'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.556183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea02c1c3fb414736a97d8896d3d6e624'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.564250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea02c1c3fb414736a97d8896d3d6e624'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.581996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1233/2485] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:16 2022] DELETE /identity/v3/users/ea02c1c3fb414736a97d8896d3d6e624 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:16.758751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:16.771156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726f73e-ec8c-4989-8bef-c1646e398bfd service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:16.833525 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:16.834183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:16.834468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:16.834743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] PATH_INFO: `/v3/projects/16f0c974e77341dea999141ee460c0c2` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:16.835857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16f0c974e77341dea999141ee460c0c2)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:16.837833 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:16.883406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16f0c974e77341dea999141ee460c0c2', 'request_id': 'req-f3821326-95d6-4583-b258-ff95c327d5d8'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.887876 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:16.889290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16f0c974e77341dea999141ee460c0c2', 'request_id': 'req-f3821326-95d6-4583-b258-ff95c327d5d8'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.899033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16f0c974e77341dea999141ee460c0c2', 'request_id': 'req-f3821326-95d6-4583-b258-ff95c327d5d8'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.911128 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] The token cache is being invalidate because project 16f0c974e77341dea999141ee460c0c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:16.911406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.912042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:16.913500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1253/2486] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:16 2022] DELETE /identity/v3/projects/16f0c974e77341dea999141ee460c0c2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:16.917501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:16.919054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:16.932036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba60a93-2cd3-4fb8-9fb9-123d965e67e8 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:16.992057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:16.992698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:16.992975 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:16.993246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] PATH_INFO: `/v3/users/be474eb968de4dd183b382b680b0ee7e` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:16.994406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] RBAC: Authorizing `identity:delete_user(user_id=be474eb968de4dd183b382b680b0ee7e)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:16.996411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.044413 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be474eb968de4dd183b382b680b0ee7e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.045054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be474eb968de4dd183b382b680b0ee7e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.052329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be474eb968de4dd183b382b680b0ee7e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.052993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be474eb968de4dd183b382b680b0ee7e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.062719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-f0d4e4c1-07cd-4905-b56d-17eef518584a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be474eb968de4dd183b382b680b0ee7e'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.081492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1234/2487] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:16 2022] DELETE /identity/v3/users/be474eb968de4dd183b382b680b0ee7e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:17.247009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.248468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:17.262635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3821326-95d6-4583-b258-ff95c327d5d8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.320541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.321197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.321475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.321776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] PATH_INFO: `/v3/projects/76004352e42d44baa87d32d0bcfb598b` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.322886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] RBAC: Authorizing `identity:delete_project(project_id=76004352e42d44baa87d32d0bcfb598b)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.324791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.370593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76004352e42d44baa87d32d0bcfb598b', 'request_id': 'req-889d1a9c-500a-4ea9-96ff-48669c6c877c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.376209 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76004352e42d44baa87d32d0bcfb598b', 'request_id': 'req-889d1a9c-500a-4ea9-96ff-48669c6c877c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.383686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76004352e42d44baa87d32d0bcfb598b', 'request_id': 'req-889d1a9c-500a-4ea9-96ff-48669c6c877c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.395146 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] The token cache is being invalidate because project 76004352e42d44baa87d32d0bcfb598b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:17.395437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.396057 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-889d1a9c-500a-4ea9-96ff-48669c6c877c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:17.397500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1254/2488] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:17 2022] DELETE /identity/v3/projects/76004352e42d44baa87d32d0bcfb598b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:17.402464 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.402753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.403036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.422723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:17.423921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:17.468030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:17.484426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1235/2489] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:17.492155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.492439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.492708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.514154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:17.515457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:17.524010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1255/2490] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:17.528542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.538256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd64fab-886f-4520-ad3c-3e01ce3df2eb None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.542723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.543311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.543586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.543856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.544492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:17.544791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.546410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.557675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1236/2491] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:17.562863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.572061 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15f603-ea9d-4300-8cc9-a8a5b58a2661 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.576714 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.577307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.577724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.578493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.579110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.581001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.609001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1256/2492] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:17 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:17.613380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.623270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f9f768-f5be-4bb5-af59-74caeaf5f6e8 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.679108 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.679706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.679985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.680250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.680871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.682818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.704805 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1237/2493] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:17 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:17.708918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.719193 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2fd9e8-02fa-4d81-97e2-c49e3f672219 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.723706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.724301 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.724579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.724848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.725480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.727102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.737207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1257/2494] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:17.741165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.755111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437e8428-2bcc-4a78-ac56-0a1bbde0a84f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.760455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aa183-7807-4f2d-9be4-eeacd97bd941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.761055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aa183-7807-4f2d-9be4-eeacd97bd941 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.761332 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aa183-7807-4f2d-9be4-eeacd97bd941 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.761632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aa183-7807-4f2d-9be4-eeacd97bd941 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.762272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aa183-7807-4f2d-9be4-eeacd97bd941 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.763875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aa183-7807-4f2d-9be4-eeacd97bd941 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.775180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1238/2495] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:17.779357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.789227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0257125-9568-4bbf-8fda-736527534a5e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:17.793830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:17.794480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.794759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.795033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] PATH_INFO: `/v3/projects/6e95409281594b7cb7d7f5d86e22f9f8/users/43ab048ca3834090972fa355a0213f24/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.810394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e95409281594b7cb7d7f5d86e22f9f8, user_id=43ab048ca3834090972fa355a0213f24, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:17.813550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:17.826520 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1258/2496] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:17 2022] PUT /identity/v3/projects/6e95409281594b7cb7d7f5d86e22f9f8/users/43ab048ca3834090972fa355a0213f24/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:17.837244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:17.837534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:17.837842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:17.865979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] MFA Rules not processed for user `43ab048ca3834090972fa355a0213f24`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:17.867192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:17.930546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1239/2497] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:53:17.934943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:17.947895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:17.998966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a0a3b-eeb4-4d5f-8aaa-7ad14b2b3415 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.004160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.004759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.005036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.005306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.005965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.007857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.033395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1259/2498] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:17 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:18.037548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.048711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a36c0a-cc49-43b6-89dd-dc94cf41649b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.089910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:18.105585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.106202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.106487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.106762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.107432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.110175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.137138 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1240/2499] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:18 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:18.138633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.150346 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e50b10-fffb-462f-a9de-57806d29c626 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.156507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.157421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.157924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.158386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.159148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.161026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.174592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1260/2500] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:18.180062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.191738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8c61fd-f093-4bf4-8209-02acf2e4b964 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.196551 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.197243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.197525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.197862 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] PATH_INFO: `/v3/projects/830ed8d203044c1281fe729cb826d913/users/8e32e92f69d94bd391c7a3cf84bbe5de/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.213718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ed8d203044c1281fe729cb826d913, user_id=8e32e92f69d94bd391c7a3cf84bbe5de, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.219794 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.234719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1241/2501] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:18 2022] PUT /identity/v3/projects/830ed8d203044c1281fe729cb826d913/users/8e32e92f69d94bd391c7a3cf84bbe5de/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:18.238788 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.248857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c385bca9-48b9-4932-ab2a-dad77a916212 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.309622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647121b-5a8a-4651-bdfe-0a327faa6459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.310232 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647121b-5a8a-4651-bdfe-0a327faa6459 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.310510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647121b-5a8a-4651-bdfe-0a327faa6459 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.310775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647121b-5a8a-4651-bdfe-0a327faa6459 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.311420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647121b-5a8a-4651-bdfe-0a327faa6459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.313038 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647121b-5a8a-4651-bdfe-0a327faa6459 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.323036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1261/2502] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:53:18.327425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.337616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa8227-4c22-47e5-b073-2a3e90398e96 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.342207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.342847 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.343126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.343923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] PATH_INFO: `/v3/projects/830ed8d203044c1281fe729cb826d913/users/8e32e92f69d94bd391c7a3cf84bbe5de/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.345899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ed8d203044c1281fe729cb826d913, user_id=8e32e92f69d94bd391c7a3cf84bbe5de, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.348523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.358897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1242/2503] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:18 2022] PUT /identity/v3/projects/830ed8d203044c1281fe729cb826d913/users/8e32e92f69d94bd391c7a3cf84bbe5de/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:18.370847 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.371160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.371435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.399310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] MFA Rules not processed for user `8e32e92f69d94bd391c7a3cf84bbe5de`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:18.400515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:18.467474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1262/2504] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:18 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:18.479185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.480861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:18.538313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076ca088-32ce-4841-ba69-0e8fa5c953c3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.541838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.542435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.542713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.542982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.552520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.553995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.563822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1243/2505] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:18 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:18.611433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:18.612823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:18.624822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8416e9e-01bd-4d98-8e09-48c5afff83b7 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:18.628469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034ac5e-96fe-43e2-9638-0f69059d8d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:18.629073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ac5e-96fe-43e2-9638-0f69059d8d89 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:18.629354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ac5e-96fe-43e2-9638-0f69059d8d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:18.629662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034ac5e-96fe-43e2-9638-0f69059d8d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:18.633912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:18.690697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034ac5e-96fe-43e2-9638-0f69059d8d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:18.692156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034ac5e-96fe-43e2-9638-0f69059d8d89 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:18.701454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1263/2506] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:21.982863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:22.043962 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6a9755-3913-4aae-ae30-bb4675bf6045 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:22.048715 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:22.049348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:22.049667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:22.049945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] PATH_INFO: `/v3/users/43ab048ca3834090972fa355a0213f24` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:22.051082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] RBAC: Authorizing `identity:delete_user(user_id=43ab048ca3834090972fa355a0213f24)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:22.053013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:22.094748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:22.099916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43ab048ca3834090972fa355a0213f24'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.100784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43ab048ca3834090972fa355a0213f24'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.108017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43ab048ca3834090972fa355a0213f24'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.109360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43ab048ca3834090972fa355a0213f24'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.116528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43ab048ca3834090972fa355a0213f24'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.133902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1244/2507] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:21 2022] DELETE /identity/v3/users/43ab048ca3834090972fa355a0213f24 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:22.138565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b8425-c590-4070-a69e-7d367a234407 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:22.139080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b8425-c590-4070-a69e-7d367a234407 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:22.139177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b8425-c590-4070-a69e-7d367a234407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:22.158103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-908b8425-c590-4070-a69e-7d367a234407 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:22.159280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-908b8425-c590-4070-a69e-7d367a234407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:22.217932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1264/2508] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:22.227580 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:22.229098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:22.288908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a04e7a0-8f36-43cb-9ebc-8afe00be471d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:22.292629 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:22.293329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:22.293755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:22.294225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:22.303152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:22.304567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:22.314306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1245/2509] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:22.578392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b8425-c590-4070-a69e-7d367a234407 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:22.579875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-908b8425-c590-4070-a69e-7d367a234407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:22.595144 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b8425-c590-4070-a69e-7d367a234407 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:22.599873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:22.601054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:22.601344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:22.601836 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] PATH_INFO: `/v3/projects/6e95409281594b7cb7d7f5d86e22f9f8` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:22.603060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e95409281594b7cb7d7f5d86e22f9f8)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:22.604973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:22.652568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e95409281594b7cb7d7f5d86e22f9f8', 'request_id': 'req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.658401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e95409281594b7cb7d7f5d86e22f9f8', 'request_id': 'req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.666001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e95409281594b7cb7d7f5d86e22f9f8', 'request_id': 'req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.677416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] The token cache is being invalidate because project 6e95409281594b7cb7d7f5d86e22f9f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:22.677812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.678442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.679909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1265/2510] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:22 2022] DELETE /identity/v3/projects/6e95409281594b7cb7d7f5d86e22f9f8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:22.684392 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:22.694826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb857da3-a32f-4855-9047-8fbb7cdd8f02 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:22.751808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:22.752431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:22.752708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:22.752982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] PATH_INFO: `/v3/users/8e32e92f69d94bd391c7a3cf84bbe5de` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:22.754121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e32e92f69d94bd391c7a3cf84bbe5de)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:22.756037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:22.802449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e32e92f69d94bd391c7a3cf84bbe5de'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.803085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e32e92f69d94bd391c7a3cf84bbe5de'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.809837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e32e92f69d94bd391c7a3cf84bbe5de'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.810469 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e32e92f69d94bd391c7a3cf84bbe5de'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.818209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7d90f7e3-0897-419a-a887-5225ac196260 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e32e92f69d94bd391c7a3cf84bbe5de'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:22.819696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:22.839044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1246/2511] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:22 2022] DELETE /identity/v3/users/8e32e92f69d94bd391c7a3cf84bbe5de => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:23.185539 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.187095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:23.200216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8b0e4-8a5b-4b2a-95d1-2f03c3063eb3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.228385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:23.260521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.261176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.261456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.261787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] PATH_INFO: `/v3/projects/830ed8d203044c1281fe729cb826d913` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.262926 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830ed8d203044c1281fe729cb826d913)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.265416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.311718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830ed8d203044c1281fe729cb826d913', 'request_id': 'req-b140b043-f906-4454-b5ef-25cd65c0ef08'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:23.317679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830ed8d203044c1281fe729cb826d913', 'request_id': 'req-b140b043-f906-4454-b5ef-25cd65c0ef08'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:23.325100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830ed8d203044c1281fe729cb826d913', 'request_id': 'req-b140b043-f906-4454-b5ef-25cd65c0ef08'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:23.336602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] The token cache is being invalidate because project 830ed8d203044c1281fe729cb826d913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:23.336885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:23.337516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b140b043-f906-4454-b5ef-25cd65c0ef08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:23.338985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1266/2512] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:23 2022] DELETE /identity/v3/projects/830ed8d203044c1281fe729cb826d913 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:23.344677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.345191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.345290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.364842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:23.366002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:23.424751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1247/2513] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:23.432737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.433023 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.433296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.454079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:23.455275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:23.463251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1267/2514] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:23.467509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.477958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b09c-4dfc-4639-aa9a-cded8dfc0f7d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.482696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.483539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.483817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.484086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.484728 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:23.485029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.486675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.498203 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1248/2515] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:23.502882 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.511856 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca78f7-098c-4e29-b135-62005f78f082 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.516925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.517512 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.517816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.518084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.518706 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.520571 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.548162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1268/2516] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:53:23.552313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.562850 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1176080-2433-4cb7-9230-0588a5944231 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.617745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.618269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.618545 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.618816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.619459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.621344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.644515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1249/2517] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:23.648730 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.660621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1feff18-5e38-4c60-a545-e96c5e876ab8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.666025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.666626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.666906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.667178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.667824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.669423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.679943 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1269/2518] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:23.683970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.694377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54def9f-3b4c-4ecf-a7f0-a62ff4b10653 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.698923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff7713d-4b62-4b81-918f-da64d1fd69de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.699488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7713d-4b62-4b81-918f-da64d1fd69de admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.699761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7713d-4b62-4b81-918f-da64d1fd69de admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.700029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7713d-4b62-4b81-918f-da64d1fd69de admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.700653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7713d-4b62-4b81-918f-da64d1fd69de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.702271 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff7713d-4b62-4b81-918f-da64d1fd69de admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.713302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1250/2519] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:23.717490 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.727654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c29c1e-3bf1-4258-9fdb-9c839fb02f15 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.732491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.733380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.733771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.734043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] PATH_INFO: `/v3/projects/ab6a711c2491420ba6627f7eac9edeb1/users/6368261635b149bb9540a7b6fb6d6f0a/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.748208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6a711c2491420ba6627f7eac9edeb1, user_id=6368261635b149bb9540a7b6fb6d6f0a, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.751456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.767049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1270/2520] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:23 2022] PUT /identity/v3/projects/ab6a711c2491420ba6627f7eac9edeb1/users/6368261635b149bb9540a7b6fb6d6f0a/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:23.777389 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.777711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.777989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.806894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] MFA Rules not processed for user `6368261635b149bb9540a7b6fb6d6f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:23.808113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:23.840346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:23.873694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1251/2521] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:23.878155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.934192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:23.939740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b470fb0b-967c-4a80-ba44-1f7d1fc160e5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:23.945486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:23.946126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:23.946400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:23.946667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:23.947282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:23.949158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:23.976163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1271/2522] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:23.980064 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:23.990994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e8eb11-0002-48b5-a17d-e1a9b82c2b4b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:24.048485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:24.049100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.049379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.049683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.050321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:24.052272 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:24.074074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1252/2523] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:23 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:24.078254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:24.088669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc5ea4-dce3-45af-b686-7494c6a128d0 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:24.093873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:24.094479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.094758 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.095026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.095656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:24.097242 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:24.108504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1272/2524] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:24.112559 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:24.124051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88182219-1839-40b6-acd0-4f8fbb723f72 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:24.131888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:24.132614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.132904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.133180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] PATH_INFO: `/v3/projects/5d63b2e2563b4ad6b46e60fdedcd97f5/users/884f8116d8d44928a22ce0434e7a5547/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.150575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d63b2e2563b4ad6b46e60fdedcd97f5, user_id=884f8116d8d44928a22ce0434e7a5547, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:24.153302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:24.167569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1253/2525] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:24 2022] PUT /identity/v3/projects/5d63b2e2563b4ad6b46e60fdedcd97f5/users/884f8116d8d44928a22ce0434e7a5547/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:24.171603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:24.182083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f7e74c-d895-4305-8946-c2df908795b1 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:24.238889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55178217-7b8f-46c0-9346-ecf5afa7dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:24.239546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55178217-7b8f-46c0-9346-ecf5afa7dcfc admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.239770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55178217-7b8f-46c0-9346-ecf5afa7dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.240046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55178217-7b8f-46c0-9346-ecf5afa7dcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.240710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55178217-7b8f-46c0-9346-ecf5afa7dcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:24.242339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55178217-7b8f-46c0-9346-ecf5afa7dcfc admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:24.252723 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1273/2526] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:24 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:24.256976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:24.268180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e0d4e1-b067-4463-b0eb-ac9deeb9f5b6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:24.272925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:24.273618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.273894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.274162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] PATH_INFO: `/v3/projects/5d63b2e2563b4ad6b46e60fdedcd97f5/users/884f8116d8d44928a22ce0434e7a5547/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.276057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d63b2e2563b4ad6b46e60fdedcd97f5, user_id=884f8116d8d44928a22ce0434e7a5547, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:24.278754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:24.289310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1254/2527] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:24 2022] PUT /identity/v3/projects/5d63b2e2563b4ad6b46e60fdedcd97f5/users/884f8116d8d44928a22ce0434e7a5547/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:24.300725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.301008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.301281 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.329453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] MFA Rules not processed for user `884f8116d8d44928a22ce0434e7a5547`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:24.330675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:24.397647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1274/2528] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:24.408337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:24.410111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:24.467479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635978b-ae6e-42e8-ad2d-ba651d8c5ba2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:24.471075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64849615-e87e-4819-966c-7014cc83dfef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:24.471667 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849615-e87e-4819-966c-7014cc83dfef service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:24.471942 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849615-e87e-4819-966c-7014cc83dfef service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:24.472211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64849615-e87e-4819-966c-7014cc83dfef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:24.514143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:24.530475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849615-e87e-4819-966c-7014cc83dfef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:24.531937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64849615-e87e-4819-966c-7014cc83dfef service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:24.540706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1255/2529] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:27.071584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:27.122002 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:27.134804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d9ee2-3b7b-4ab4-ba24-74c11f055f2d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:27.139252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:27.140430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:27.140699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:27.141084 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] PATH_INFO: `/v3/users/6368261635b149bb9540a7b6fb6d6f0a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:27.142203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6368261635b149bb9540a7b6fb6d6f0a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:27.144110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:27.190728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6368261635b149bb9540a7b6fb6d6f0a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.191378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6368261635b149bb9540a7b6fb6d6f0a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.199091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6368261635b149bb9540a7b6fb6d6f0a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.199869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6368261635b149bb9540a7b6fb6d6f0a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.212476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6368261635b149bb9540a7b6fb6d6f0a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.224927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1275/2530] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:27 2022] DELETE /identity/v3/users/6368261635b149bb9540a7b6fb6d6f0a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:27.229446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:27.229964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:27.230048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:27.249774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:27.251021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:27.311013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1256/2531] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:27.321247 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:27.322696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:27.379334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be766bd0-db68-4b40-94e3-75b20104ad76 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:27.382990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:27.383593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:27.383875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:27.384150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:27.394441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:27.395866 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:27.405267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1276/2532] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:27.728398 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:27.729981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:27.760591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0357f4-c0cd-4af4-a551-a987079e54f6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:27.773686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:27.774334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:27.774607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:27.774875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] PATH_INFO: `/v3/projects/ab6a711c2491420ba6627f7eac9edeb1` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:27.776005 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6a711c2491420ba6627f7eac9edeb1)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:27.777968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:27.831282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6a711c2491420ba6627f7eac9edeb1', 'request_id': 'req-706a4eef-c8be-478d-9636-d22fc3be7a12'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.836934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6a711c2491420ba6627f7eac9edeb1', 'request_id': 'req-706a4eef-c8be-478d-9636-d22fc3be7a12'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.844519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6a711c2491420ba6627f7eac9edeb1', 'request_id': 'req-706a4eef-c8be-478d-9636-d22fc3be7a12'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.858414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] The token cache is being invalidate because project ab6a711c2491420ba6627f7eac9edeb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:27.858752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.859508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:27.861076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1257/2533] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:27 2022] DELETE /identity/v3/projects/ab6a711c2491420ba6627f7eac9edeb1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:27.865266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:27.877153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab6d46-ba68-4cbb-928d-b480e1849383 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:27.942272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:27.942940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:27.943296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:27.943647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] PATH_INFO: `/v3/users/884f8116d8d44928a22ce0434e7a5547` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:27.944863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=884f8116d8d44928a22ce0434e7a5547)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:27.946975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:27.963510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:28.004341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '884f8116d8d44928a22ce0434e7a5547'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.005154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '884f8116d8d44928a22ce0434e7a5547'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.013867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '884f8116d8d44928a22ce0434e7a5547'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.015593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '884f8116d8d44928a22ce0434e7a5547'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.031320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-39b5ee30-cdf4-412d-a895-2158902b8b29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '884f8116d8d44928a22ce0434e7a5547'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.047396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1277/2534] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:27 2022] DELETE /identity/v3/users/884f8116d8d44928a22ce0434e7a5547 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:28.347129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.348621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:28.364056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706a4eef-c8be-478d-9636-d22fc3be7a12 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.429792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.430476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.430755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.431054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] PATH_INFO: `/v3/projects/5d63b2e2563b4ad6b46e60fdedcd97f5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.432263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d63b2e2563b4ad6b46e60fdedcd97f5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.434243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.444358 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:28.488693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d63b2e2563b4ad6b46e60fdedcd97f5', 'request_id': 'req-3298ac68-6354-4afe-b19e-892601ce42fa'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.494492 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d63b2e2563b4ad6b46e60fdedcd97f5', 'request_id': 'req-3298ac68-6354-4afe-b19e-892601ce42fa'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.503154 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d63b2e2563b4ad6b46e60fdedcd97f5', 'request_id': 'req-3298ac68-6354-4afe-b19e-892601ce42fa'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.517113 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] The token cache is being invalidate because project 5d63b2e2563b4ad6b46e60fdedcd97f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:28.517548 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.518324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-3298ac68-6354-4afe-b19e-892601ce42fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:28.519873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1258/2535] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:28 2022] DELETE /identity/v3/projects/5d63b2e2563b4ad6b46e60fdedcd97f5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:28.525008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.525503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.525722 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.545781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:28.547469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:28.607995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1278/2536] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:28.617050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a0170-8049-424a-acc6-d78895d87401 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.617330 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a0170-8049-424a-acc6-d78895d87401 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.617628 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a0170-8049-424a-acc6-d78895d87401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.639118 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-538a0170-8049-424a-acc6-d78895d87401 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:28.640596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-538a0170-8049-424a-acc6-d78895d87401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:28.648996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1259/2537] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:28.653259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.663563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a0f279-d7a8-47dc-b579-d0af48d40ace None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.668342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.668955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.669234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.669506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.670308 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:28.670620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.672899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.684757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1279/2538] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:53:28.689934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a0170-8049-424a-acc6-d78895d87401 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.699396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a0170-8049-424a-acc6-d78895d87401 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.704046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.704673 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.704952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.705223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.705864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.707766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.738172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1260/2539] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:28 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:28.742390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.754940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654e85a-d36c-48fc-a90b-1fe29323584d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.820372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.822134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.822505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.822827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.823462 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.825980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.850404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1280/2540] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:28 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:28.854557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.866360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15fd960-fc55-4880-ba5b-4d7cf35fd5b6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.871023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.871687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.871902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.872220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.872855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.874480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.884367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1261/2541] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:53:28.888551 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.898554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34f0a6-9700-46a1-984b-4464c3f84f1b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.903083 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fe0e0a-0023-4373-a1b3-39718d904c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.903699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0e0a-0023-4373-a1b3-39718d904c05 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.903986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0e0a-0023-4373-a1b3-39718d904c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.904234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fe0e0a-0023-4373-a1b3-39718d904c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.904861 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe0e0a-0023-4373-a1b3-39718d904c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.907024 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fe0e0a-0023-4373-a1b3-39718d904c05 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.909202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:28.919589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1281/2542] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:28.924348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:28.934641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac14b14d-b5c1-4993-8ddb-27b44383e309 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:28.939261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:28.939900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.940183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.940459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] PATH_INFO: `/v3/projects/532adcdeca994762991de3a7511c917d/users/605822c1283041719609385322a7a7ce/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:28.955219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] RBAC: Authorizing `identity:create_grant(project_id=532adcdeca994762991de3a7511c917d, user_id=605822c1283041719609385322a7a7ce, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:28.958085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:28.972090 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1262/2543] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:28 2022] PUT /identity/v3/projects/532adcdeca994762991de3a7511c917d/users/605822c1283041719609385322a7a7ce/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:28.982493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:28.982777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:28.983049 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:29.011172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] MFA Rules not processed for user `605822c1283041719609385322a7a7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:29.012337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:29.078074 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1282/2544] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:29.090575 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:29.092296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:29.152614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809d648-dc8c-4144-86bf-1bfb38351ffb admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:29.156864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9356dae8-456a-4217-b0ef-c7befac15da2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:29.157470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356dae8-456a-4217-b0ef-c7befac15da2 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:29.157768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356dae8-456a-4217-b0ef-c7befac15da2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:29.158038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9356dae8-456a-4217-b0ef-c7befac15da2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:29.167163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356dae8-456a-4217-b0ef-c7befac15da2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:29.168647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9356dae8-456a-4217-b0ef-c7befac15da2 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:29.177880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1263/2545] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:39.073970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=33 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:39.173771 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:47.745685 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:47.810494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271fe14-04f4-45d7-a105-c33d6596e744 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:47.815712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:47.816373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:47.816652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:47.816924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] PATH_INFO: `/v3/users/605822c1283041719609385322a7a7ce` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:47.818081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=605822c1283041719609385322a7a7ce)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:47.820000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:47.864870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '605822c1283041719609385322a7a7ce'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:47.865517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '605822c1283041719609385322a7a7ce'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:47.873705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '605822c1283041719609385322a7a7ce'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:47.874339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '605822c1283041719609385322a7a7ce'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:47.886613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '605822c1283041719609385322a7a7ce'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:47.899248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1283/2546] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:47 2022] DELETE /identity/v3/users/605822c1283041719609385322a7a7ce => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:47.903762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:47.904252 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:47.904359 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:47.923865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:47.925112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:47.982811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1264/2547] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:47.992678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:47.994113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:48.050382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f61c56-384b-4525-84e4-80c485131849 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:48.054036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b43ea9-a104-4585-8992-0cb6bd2f0f07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:48.054635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43ea9-a104-4585-8992-0cb6bd2f0f07 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.054916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43ea9-a104-4585-8992-0cb6bd2f0f07 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.055188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b43ea9-a104-4585-8992-0cb6bd2f0f07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.064830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43ea9-a104-4585-8992-0cb6bd2f0f07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:48.066930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b43ea9-a104-4585-8992-0cb6bd2f0f07 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:48.076715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1284/2548] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:48.597255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:48.598844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:48.616434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc7a01-0e9d-4fa0-aecd-55c431441ea1 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:48.621661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:48.622339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.622622 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.622993 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] PATH_INFO: `/v3/projects/532adcdeca994762991de3a7511c917d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.624024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=532adcdeca994762991de3a7511c917d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:48.626010 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:48.674652 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '532adcdeca994762991de3a7511c917d', 'request_id': 'req-6e50c376-cdf3-4560-99af-6937681d7350'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:48.680183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '532adcdeca994762991de3a7511c917d', 'request_id': 'req-6e50c376-cdf3-4560-99af-6937681d7350'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:48.687779 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '532adcdeca994762991de3a7511c917d', 'request_id': 'req-6e50c376-cdf3-4560-99af-6937681d7350'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:48.700518 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] The token cache is being invalidate because project 532adcdeca994762991de3a7511c917d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:48.700807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:48.701490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6e50c376-cdf3-4560-99af-6937681d7350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:48.703022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1265/2549] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:48 2022] DELETE /identity/v3/projects/532adcdeca994762991de3a7511c917d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:48.707969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.708251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.708530 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.728217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:48.729429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:48.748123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:48.791020 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1285/2550] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:48.799524 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.799824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.800173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.819817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:48.821036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:48.829869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1266/2551] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:48.834091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:48.843810 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68daddd9-9036-404c-b444-5b58d037cc12 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:48.848883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:48.849484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.849805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.850079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.850728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:48.851033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:48.852633 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:48.865080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1286/2552] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:48.869451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:48.878918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ea35c-1b4e-42e5-8e80-853b0ebad0b9 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:48.883572 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:48.884197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.884476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.884783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.885426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:48.887371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:48.915119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1267/2553] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:48 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:48.919474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:48.930264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea69f96-f99f-4764-93ea-222351619bdd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:48.990559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:48.991171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:48.991451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:48.991720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:48.992355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:48.994322 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:49.015248 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1287/2554] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:48 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:49.019983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:49.031705 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9cdf78-17b1-4166-8fe3-e7faf92de308 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:49.037184 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:49.037817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:49.038097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:49.038442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:49.039086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:49.040984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:49.055029 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1268/2555] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:49.056215 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:49.066293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44753b2c-0b58-494b-8b5b-0c7a82c1dd0e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:49.070840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446e73f-5ff5-4170-b70b-cbf7a325e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:49.071435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446e73f-5ff5-4170-b70b-cbf7a325e619 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:49.071710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446e73f-5ff5-4170-b70b-cbf7a325e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:49.071978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446e73f-5ff5-4170-b70b-cbf7a325e619 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:49.072612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446e73f-5ff5-4170-b70b-cbf7a325e619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:49.074241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446e73f-5ff5-4170-b70b-cbf7a325e619 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:49.085591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1288/2556] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:49.089858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:49.101188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4ef44-2915-42ff-aeb7-5d43eda64fd3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:49.106085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:49.106741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:49.107023 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:49.107414 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] PATH_INFO: `/v3/projects/66045662d0314358b4ba5fa791f539ce/users/66248f7692cc46a0afeea0f0368c31f8/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:49.122964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] RBAC: Authorizing `identity:create_grant(project_id=66045662d0314358b4ba5fa791f539ce, user_id=66248f7692cc46a0afeea0f0368c31f8, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:49.125647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:49.139752 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1269/2557] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:49 2022] PUT /identity/v3/projects/66045662d0314358b4ba5fa791f539ce/users/66248f7692cc46a0afeea0f0368c31f8/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:49.151141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:49.151141 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:49.151475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:49.179176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] MFA Rules not processed for user `66248f7692cc46a0afeea0f0368c31f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:49.180429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:49.246754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1289/2558] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:49.256377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:49.258149 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:49.284966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:49.324089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa977248-34a5-4e57-9a5a-f8d1ac80559c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:49.328574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:49.329826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:49.330114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:49.330383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:49.340269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:49.341740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:49.350763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1270/2559] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:53.812818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:53.814883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:53.835891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:53.906034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45402359-3878-41f3-afd7-27a002e56b5c None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:53.909655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:53.910269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:53.910547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:53.910817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:53.912555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:54.019606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:54.021087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:54.041238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1290/2560] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:53:57.106615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.108354 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:57.140170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9670945a-b347-43ec-bdda-b9a1fb0069f8 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.144650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.145267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.145542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.145845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] PATH_INFO: `/v3/users/0b18352a23444324b27d17d2cee636fc` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.146954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b18352a23444324b27d17d2cee636fc)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.148854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:57.194920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b18352a23444324b27d17d2cee636fc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.195503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b18352a23444324b27d17d2cee636fc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.202282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b18352a23444324b27d17d2cee636fc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.202908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b18352a23444324b27d17d2cee636fc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.210609 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ed6ee3eb-5ffe-49f0-8808-936b34789230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b18352a23444324b27d17d2cee636fc'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.228463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1271/2561] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:57 2022] DELETE /identity/v3/users/0b18352a23444324b27d17d2cee636fc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:57.407166 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.420042 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e506894c-d7c6-4af8-a059-501057766f34 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.489098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=100 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:57.499735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.500394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.500749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.501015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] PATH_INFO: `/v3/projects/47156a6d045b4147a837fbf132421a53` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.502343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47156a6d045b4147a837fbf132421a53)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.504473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:57.557118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47156a6d045b4147a837fbf132421a53', 'request_id': 'req-3bb56d52-e012-4897-803f-7b72f3323207'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.564419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47156a6d045b4147a837fbf132421a53', 'request_id': 'req-3bb56d52-e012-4897-803f-7b72f3323207'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.571989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47156a6d045b4147a837fbf132421a53', 'request_id': 'req-3bb56d52-e012-4897-803f-7b72f3323207'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.583822 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] The token cache is being invalidate because project 47156a6d045b4147a837fbf132421a53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:57.584109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.584734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3bb56d52-e012-4897-803f-7b72f3323207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:57.586225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1291/2562] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:57 2022] DELETE /identity/v3/projects/47156a6d045b4147a837fbf132421a53 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:57.591136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.591424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.591701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.649411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:57.650674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:57.692496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:57.717793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1272/2563] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:53:57.720594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.720880 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.721153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.742026 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:57.743268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:57.752830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1292/2564] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:57.757212 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.768963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a0c9-b943-41df-bfe6-7cd6e4c76bb8 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.774034 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.774691 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.775003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.775304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.776546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:53:57.776546 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.778040 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:57.790342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1273/2565] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:53:57.794727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.804406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2affc8-1b73-418e-bf07-5a3a72f0c88d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.809063 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.809688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.809970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.810243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.810877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.812770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:57.841803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1293/2566] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:57 2022] POST /identity/v3/projects => generated 405 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:53:57.846394 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.856411 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899d29c-64c9-471e-893b-ad1cf1fb7fc4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.912860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.913463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.913774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.914051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.914682 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.916617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:57.937148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1274/2567] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:57 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:53:57.942135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.952274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bd05af-88b0-4a13-88e6-479286e8dc43 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.957155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.957777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.958059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.958345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.958992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.960610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:57.971413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1294/2568] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:57.975347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:57.986117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ac9d-e496-487e-b66b-bbd101eebae9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:57.990724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be4b50-e109-4b66-90b9-8b692f63ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:57.991325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be4b50-e109-4b66-90b9-8b692f63ea51 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:57.991607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be4b50-e109-4b66-90b9-8b692f63ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:57.991884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be4b50-e109-4b66-90b9-8b692f63ea51 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:57.992527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be4b50-e109-4b66-90b9-8b692f63ea51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:57.994172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be4b50-e109-4b66-90b9-8b692f63ea51 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.005137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1275/2569] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:58.009409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.019401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60f41-a84b-4d39-b044-7394713e2361 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.023978 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.024621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.024899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.025172 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] PATH_INFO: `/v3/projects/015a3b4b888d4ae3bbc6b61c9b3c0a61/users/9be042e34af143018cf6c9df031206e0/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.039501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=015a3b4b888d4ae3bbc6b61c9b3c0a61, user_id=9be042e34af143018cf6c9df031206e0, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.042192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.055778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1295/2570] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:58 2022] PUT /identity/v3/projects/015a3b4b888d4ae3bbc6b61c9b3c0a61/users/9be042e34af143018cf6c9df031206e0/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:58.066765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.067055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.067335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.094201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] MFA Rules not processed for user `9be042e34af143018cf6c9df031206e0`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:58.095418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:58.160347 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1276/2571] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:58 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:58.164912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.226803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf67ca5-88f0-4c39-9a80-9ca1c83ae550 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.231456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.232054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.232334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.232607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.233489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.235932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.263333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1296/2572] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:53:58 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:58.267362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.278468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5959d08-e896-4c92-92d1-3561420530d2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.320276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:58.338243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.338853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.339131 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.339402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.340030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.341985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.366061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1277/2573] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:53:58 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:53:58.369908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.375345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:58.381884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25eac4-fd6a-4f0e-968b-6469e799c807 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.387886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.388493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.388772 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.389041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.389743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.391338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.402009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1297/2574] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:58.406161 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.416686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc48a2e-90a0-4ca9-bf22-aea17c3fef23 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.421182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.421925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.422202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.422473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] PATH_INFO: `/v3/projects/9abcd8bac5634f40a737516f1d44193b/users/8013902dd7ec4601aabdc2f4c1d32626/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.437438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abcd8bac5634f40a737516f1d44193b, user_id=8013902dd7ec4601aabdc2f4c1d32626, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.440099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.453022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1278/2575] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:58 2022] PUT /identity/v3/projects/9abcd8bac5634f40a737516f1d44193b/users/8013902dd7ec4601aabdc2f4c1d32626/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:58.457091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.467165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0444b2-73ea-4589-b1d8-8b7d1469ed92 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.523455 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409622a7-d34a-494b-aa87-693e8d05cd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.523980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409622a7-d34a-494b-aa87-693e8d05cd25 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.524261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409622a7-d34a-494b-aa87-693e8d05cd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.524533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409622a7-d34a-494b-aa87-693e8d05cd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.525253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409622a7-d34a-494b-aa87-693e8d05cd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.526872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409622a7-d34a-494b-aa87-693e8d05cd25 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.537470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1298/2576] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:53:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:53:58.541669 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.552552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f133f684-bc91-4b81-aa90-de95a8563c6c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.557220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.557897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.558175 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.558448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] PATH_INFO: `/v3/projects/9abcd8bac5634f40a737516f1d44193b/users/8013902dd7ec4601aabdc2f4c1d32626/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.560363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abcd8bac5634f40a737516f1d44193b, user_id=8013902dd7ec4601aabdc2f4c1d32626, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.563030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.574016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1279/2577] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:53:58 2022] PUT /identity/v3/projects/9abcd8bac5634f40a737516f1d44193b/users/8013902dd7ec4601aabdc2f4c1d32626/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:58.585304 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.585609 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.585889 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.616731 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] MFA Rules not processed for user `8013902dd7ec4601aabdc2f4c1d32626`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:58.618195 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:58.699918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1299/2578] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:58 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:58.711352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:58.713289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:58.774922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb54a4-6554-42db-9234-4f3c7bc98aae admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:58.778449 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffce3cf4-a58e-4447-a271-12f2fe60f328 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:58.779049 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce3cf4-a58e-4447-a271-12f2fe60f328 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:58.779321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce3cf4-a58e-4447-a271-12f2fe60f328 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:58.779588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffce3cf4-a58e-4447-a271-12f2fe60f328 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:58.789112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce3cf4-a58e-4447-a271-12f2fe60f328 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:58.790564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffce3cf4-a58e-4447-a271-12f2fe60f328 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:58.800066 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1280/2579] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:58 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:53:58.961201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:59.025290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3372d48-8ca1-48c5-a55c-7b3cf78befac None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:59.029728 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:59.030459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.030742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.031016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] PATH_INFO: `/v3/users/9be042e34af143018cf6c9df031206e0` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.032843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9be042e34af143018cf6c9df031206e0)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:59.034812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:59.048715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:59.083150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9be042e34af143018cf6c9df031206e0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.083799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9be042e34af143018cf6c9df031206e0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.091271 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9be042e34af143018cf6c9df031206e0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.091872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9be042e34af143018cf6c9df031206e0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.105050 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9be042e34af143018cf6c9df031206e0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.121486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1300/2580] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:58 2022] DELETE /identity/v3/users/9be042e34af143018cf6c9df031206e0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:59.126676 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.127030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.127335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.146094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:59.147799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:59.208892 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1281/2581] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:53:59.218726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:59.220212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:59.278307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65798093-beb4-4a9c-afe1-5871909db114 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:59.281932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:59.282537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.282817 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.283087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.292119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:59.293581 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:59.302863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1301/2582] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:53:59.570937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:59.572388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:59.585155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d4018-1e1f-49b3-bd2a-7bdd4125910e None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:59.589935 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:59.591140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.591434 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.591731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] PATH_INFO: `/v3/projects/015a3b4b888d4ae3bbc6b61c9b3c0a61` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.593292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=015a3b4b888d4ae3bbc6b61c9b3c0a61)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:59.595208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:59.631557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:59.644192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015a3b4b888d4ae3bbc6b61c9b3c0a61', 'request_id': 'req-86348dae-8cd6-4cf3-9c08-17768aed71c2'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.649522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015a3b4b888d4ae3bbc6b61c9b3c0a61', 'request_id': 'req-86348dae-8cd6-4cf3-9c08-17768aed71c2'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.655569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015a3b4b888d4ae3bbc6b61c9b3c0a61', 'request_id': 'req-86348dae-8cd6-4cf3-9c08-17768aed71c2'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.669338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] The token cache is being invalidate because project 015a3b4b888d4ae3bbc6b61c9b3c0a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:53:59.669675 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.670306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.671809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1282/2583] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:53:59 2022] DELETE /identity/v3/projects/015a3b4b888d4ae3bbc6b61c9b3c0a61 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:59.676239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:59.686299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f9956-68c6-4386-b06c-d2f68b3b33fd service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:59.723126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:53:59.724532 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:53:59.746661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:59.747293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.747572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.747843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] PATH_INFO: `/v3/users/8013902dd7ec4601aabdc2f4c1d32626` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.748981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8013902dd7ec4601aabdc2f4c1d32626)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:59.751475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:59.792867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:53:59.794982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86348dae-8cd6-4cf3-9c08-17768aed71c2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:53:59.800774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:53:59.801425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.801726 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.802033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] PATH_INFO: `/v3/users/66248f7692cc46a0afeea0f0368c31f8` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.803144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=66248f7692cc46a0afeea0f0368c31f8)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:53:59.804885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8013902dd7ec4601aabdc2f4c1d32626'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.805080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:53:59.805905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8013902dd7ec4601aabdc2f4c1d32626'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.815279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8013902dd7ec4601aabdc2f4c1d32626'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.815960 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8013902dd7ec4601aabdc2f4c1d32626'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.829805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-8f3448c0-841a-402c-9ea2-156002ce5934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8013902dd7ec4601aabdc2f4c1d32626'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.846135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1302/2584] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:59 2022] DELETE /identity/v3/users/8013902dd7ec4601aabdc2f4c1d32626 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:53:59.859539 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66248f7692cc46a0afeea0f0368c31f8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.860952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66248f7692cc46a0afeea0f0368c31f8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.869931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66248f7692cc46a0afeea0f0368c31f8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.870880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66248f7692cc46a0afeea0f0368c31f8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.881408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66248f7692cc46a0afeea0f0368c31f8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:53:59.904912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1283/2585] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:53:59 2022] DELETE /identity/v3/users/66248f7692cc46a0afeea0f0368c31f8 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:53:59.911158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:53:59.911441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:53:59.911710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:53:59.934464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:53:59.935686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.009164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1303/2586] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:00.022711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.024105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.076120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:00.100287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9080b5-3a2f-4f62-a962-11f82053b9ea admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.105291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee31794c-226f-4382-a3f6-4891bc43fed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.106019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31794c-226f-4382-a3f6-4891bc43fed7 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.106352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31794c-226f-4382-a3f6-4891bc43fed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.106613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee31794c-226f-4382-a3f6-4891bc43fed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.119820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31794c-226f-4382-a3f6-4891bc43fed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.121275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee31794c-226f-4382-a3f6-4891bc43fed7 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.132756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1284/2587] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:00.195930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.198498 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.217150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea3bd1-bc53-4afa-9064-66352b574ea5 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.234098 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.235253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.235432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.235643 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] PATH_INFO: `/v3/projects/9abcd8bac5634f40a737516f1d44193b` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.239127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9abcd8bac5634f40a737516f1d44193b)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.243483 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.330031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9abcd8bac5634f40a737516f1d44193b', 'request_id': 'req-94646229-2407-4eba-88ea-e510296733a8'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.338975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9abcd8bac5634f40a737516f1d44193b', 'request_id': 'req-94646229-2407-4eba-88ea-e510296733a8'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.352044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9abcd8bac5634f40a737516f1d44193b', 'request_id': 'req-94646229-2407-4eba-88ea-e510296733a8'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.371850 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] The token cache is being invalidate because project 9abcd8bac5634f40a737516f1d44193b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:00.372435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.373365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.375377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1304/2588] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:00 2022] DELETE /identity/v3/projects/9abcd8bac5634f40a737516f1d44193b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:00.382527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3426-9b0b-4cf9-986d-9d0475e26446 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.383027 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3426-9b0b-4cf9-986d-9d0475e26446 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.383257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e3426-9b0b-4cf9-986d-9d0475e26446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.403712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0d1e3426-9b0b-4cf9-986d-9d0475e26446 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:00.404945 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0d1e3426-9b0b-4cf9-986d-9d0475e26446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.492069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.493843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.502028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1285/2589] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:00.511030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.511316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.511592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.520441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94646229-2407-4eba-88ea-e510296733a8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.528989 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.537868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:00.541055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.541448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.541783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.542068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] PATH_INFO: `/v3/projects/66045662d0314358b4ba5fa791f539ce` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.543366 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=66045662d0314358b4ba5fa791f539ce)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.545503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.551499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1286/2590] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:00.556046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.568263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf0be8c-5749-411c-9908-be082ceb8dda None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.573812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.574692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.574986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.575260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.575912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:00.576218 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.577970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.587990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:00.591516 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1287/2591] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:00.596294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.602086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66045662d0314358b4ba5fa791f539ce', 'request_id': 'req-bbdba4f2-f911-493b-b129-86fe1241b7bd'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.606487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5da033-5afd-487b-a0ee-0a6f183ffddd admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.609153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66045662d0314358b4ba5fa791f539ce', 'request_id': 'req-bbdba4f2-f911-493b-b129-86fe1241b7bd'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.618934 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66045662d0314358b4ba5fa791f539ce', 'request_id': 'req-bbdba4f2-f911-493b-b129-86fe1241b7bd'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.631257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] The token cache is being invalidate because project 66045662d0314358b4ba5fa791f539ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:00.631629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.632354 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:00.633946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1305/2592] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:00 2022] DELETE /identity/v3/projects/66045662d0314358b4ba5fa791f539ce => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:00.670495 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.671281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.671626 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.671973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.672750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.674890 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.710519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1288/2593] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:00 2022] POST /identity/v3/projects => generated 355 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:54:00.714678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.716287 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:00.729759 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdba4f2-f911-493b-b129-86fe1241b7bd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.793272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.793930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.794213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.794484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.795115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.797100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.823973 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1306/2594] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:00 2022] POST /identity/v3/users => generated 278 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:00.828380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.839960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244bf6ef-4038-4a00-84bb-927637f2411d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.844941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.845595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.845879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.846150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.846800 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.848424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.860208 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1289/2595] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:00.864776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.875246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299c907-e685-4627-8a67-103e17d1df78 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.880799 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00650b6a-8573-49d9-8bdc-1cc5fb2b09d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.881404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00650b6a-8573-49d9-8bdc-1cc5fb2b09d8 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.881718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00650b6a-8573-49d9-8bdc-1cc5fb2b09d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.882017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00650b6a-8573-49d9-8bdc-1cc5fb2b09d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.882678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00650b6a-8573-49d9-8bdc-1cc5fb2b09d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.884288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00650b6a-8573-49d9-8bdc-1cc5fb2b09d8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.895583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1307/2596] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:54:00.899763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:00.909745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e524f36f-1771-400c-878e-69d636a9a37a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:00.914291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:00.914928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.915202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.915470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] PATH_INFO: `/v3/projects/7564bf84156d41938c3c48f7941b4222/users/d3585f8a0d02485d971f8488ca42ed12/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.930658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7564bf84156d41938c3c48f7941b4222, user_id=d3585f8a0d02485d971f8488ca42ed12, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:00.933327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:00.947032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1290/2597] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:00 2022] PUT /identity/v3/projects/7564bf84156d41938c3c48f7941b4222/users/d3585f8a0d02485d971f8488ca42ed12/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:00.957284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:00.957594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:00.957884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:00.986500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] MFA Rules not processed for user `d3585f8a0d02485d971f8488ca42ed12`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:00.987196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:01.060478 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1308/2598] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:00 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:01.065708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.080592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:01.131762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66cd90-9761-420c-bc7a-928df05e6c1a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.136416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.137016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.137295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.137578 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.138211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.140114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.166176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1291/2599] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:01 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:01.170360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.181543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40e9753-8e8a-4e30-8fa6-b8daab7c0f68 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.207069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:01.240009 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.240621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.240906 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.241183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.241842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.243859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.267362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1309/2600] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:01 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:01.271428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.281690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e2c00-0b03-418f-827d-2ef4187c14d5 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.286194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.286781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.287055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.287320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.287950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.289577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.300711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1292/2601] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:01.304676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.315372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f7d21-2254-4f2a-9446-948ef9b1e3b6 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.319965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.320606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.320881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.321167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] PATH_INFO: `/v3/projects/999c945890d6427a8c9f4b7022b1a4ed/users/cb60b79de87843d4bc31cc46f5fdb263/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.335585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=999c945890d6427a8c9f4b7022b1a4ed, user_id=cb60b79de87843d4bc31cc46f5fdb263, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.338199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.351750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1310/2602] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:01 2022] PUT /identity/v3/projects/999c945890d6427a8c9f4b7022b1a4ed/users/cb60b79de87843d4bc31cc46f5fdb263/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:01.355712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.366781 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13d483f-625e-4be3-82e9-d22f09d37268 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.422357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1ae9c6-e380-458f-a4b0-332f99c5c10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.423639 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae9c6-e380-458f-a4b0-332f99c5c10c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.423748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae9c6-e380-458f-a4b0-332f99c5c10c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.424019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1ae9c6-e380-458f-a4b0-332f99c5c10c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.424878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae9c6-e380-458f-a4b0-332f99c5c10c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.426502 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1ae9c6-e380-458f-a4b0-332f99c5c10c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.436186 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1293/2603] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:01.440300 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.450239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a20b4-1a86-4b67-8ce9-78fcdd328be8 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.454777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.455421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.455695 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.455964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] PATH_INFO: `/v3/projects/999c945890d6427a8c9f4b7022b1a4ed/users/cb60b79de87843d4bc31cc46f5fdb263/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.457869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=999c945890d6427a8c9f4b7022b1a4ed, user_id=cb60b79de87843d4bc31cc46f5fdb263, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.460475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.470826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1311/2604] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:01 2022] PUT /identity/v3/projects/999c945890d6427a8c9f4b7022b1a4ed/users/cb60b79de87843d4bc31cc46f5fdb263/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:01.481815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.482148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.482445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.510584 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] MFA Rules not processed for user `cb60b79de87843d4bc31cc46f5fdb263`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:01.511797 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:01.579811 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1294/2605] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:01.591161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.592803 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:01.649813 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e75964-ff28-4a65-bea5-31ad51573d00 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.653341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.654066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.654342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.654639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.665430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.667103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.677397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1312/2606] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:01.752348 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:01.762690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f50920-d389-4bdf-84e8-0710bd7ed533 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:01.766732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d630257-a45c-4226-9323-464333ed0c5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:01.767326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d630257-a45c-4226-9323-464333ed0c5a service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:01.767600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d630257-a45c-4226-9323-464333ed0c5a service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:01.767863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d630257-a45c-4226-9323-464333ed0c5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:01.773004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:01.824477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d630257-a45c-4226-9323-464333ed0c5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:01.826194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d630257-a45c-4226-9323-464333ed0c5a service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:01.835516 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1295/2607] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:11.672474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:11.831528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=24 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:16.648257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:16.711915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5749d750-51e6-4bc1-bff9-c3706d161b8d service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:16.716442 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:16.717077 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:16.717344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:16.717691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] PATH_INFO: `/v3/users/d3585f8a0d02485d971f8488ca42ed12` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:16.718807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3585f8a0d02485d971f8488ca42ed12)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:16.721647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:16.768372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3585f8a0d02485d971f8488ca42ed12'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:16.769091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3585f8a0d02485d971f8488ca42ed12'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:16.776401 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3585f8a0d02485d971f8488ca42ed12'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:16.777025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3585f8a0d02485d971f8488ca42ed12'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:16.789420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3585f8a0d02485d971f8488ca42ed12'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:16.802123 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1313/2608] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:16 2022] DELETE /identity/v3/users/d3585f8a0d02485d971f8488ca42ed12 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:16.806515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:16.806983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:16.807112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:16.825842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:16.827050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:16.886649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1296/2609] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:16.896969 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:16.898453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:16.975699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d251ab-7b47-4826-8ec3-07e21263dd11 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:16.979421 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:16.980022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:16.980300 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:16.980570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:16.990216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:16.991651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:17.001227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1314/2610] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:17.331291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:17.332816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:17.344661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8cd26-defa-455d-ba47-3017d9b5be71 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:17.349343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:17.350941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:17.351564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:17.351855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] PATH_INFO: `/v3/projects/7564bf84156d41938c3c48f7941b4222` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:17.353026 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7564bf84156d41938c3c48f7941b4222)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:17.355885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:17.407313 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7564bf84156d41938c3c48f7941b4222', 'request_id': 'req-7202aa7b-35e5-4da3-b988-105c9f97e446'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.412819 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7564bf84156d41938c3c48f7941b4222', 'request_id': 'req-7202aa7b-35e5-4da3-b988-105c9f97e446'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.419858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7564bf84156d41938c3c48f7941b4222', 'request_id': 'req-7202aa7b-35e5-4da3-b988-105c9f97e446'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.432730 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] The token cache is being invalidate because project 7564bf84156d41938c3c48f7941b4222 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:17.433046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.433712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.435197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1297/2611] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:17 2022] DELETE /identity/v3/projects/7564bf84156d41938c3c48f7941b4222 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:17.439230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:17.444654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:17.450839 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1ab0c-7382-4343-b86c-84e4aef5604d service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:17.507071 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:17.507689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:17.507967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:17.508239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] PATH_INFO: `/v3/users/cb60b79de87843d4bc31cc46f5fdb263` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:17.509371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb60b79de87843d4bc31cc46f5fdb263)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:17.511303 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:17.557012 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb60b79de87843d4bc31cc46f5fdb263'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.557724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb60b79de87843d4bc31cc46f5fdb263'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.565716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb60b79de87843d4bc31cc46f5fdb263'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.566345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb60b79de87843d4bc31cc46f5fdb263'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.578383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c2a95d3b-4905-4a8e-a92f-846bc2e939de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb60b79de87843d4bc31cc46f5fdb263'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:17.590544 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1315/2612] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:17 2022] DELETE /identity/v3/users/cb60b79de87843d4bc31cc46f5fdb263 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:54:17.867019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:17.868465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:17.883637 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7202aa7b-35e5-4da3-b988-105c9f97e446 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:17.942117 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:17.942656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:17.942939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:17.943209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] PATH_INFO: `/v3/projects/999c945890d6427a8c9f4b7022b1a4ed` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:17.944320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] RBAC: Authorizing `identity:delete_project(project_id=999c945890d6427a8c9f4b7022b1a4ed)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:17.946260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:17.976011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:17.994592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999c945890d6427a8c9f4b7022b1a4ed', 'request_id': 'req-a64456d5-089f-4176-a980-3d3f7b71255d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:18.000003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999c945890d6427a8c9f4b7022b1a4ed', 'request_id': 'req-a64456d5-089f-4176-a980-3d3f7b71255d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:18.006024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999c945890d6427a8c9f4b7022b1a4ed', 'request_id': 'req-a64456d5-089f-4176-a980-3d3f7b71255d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:18.018950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] The token cache is being invalidate because project 999c945890d6427a8c9f4b7022b1a4ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:18.019242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:18.019875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-a64456d5-089f-4176-a980-3d3f7b71255d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:18.021351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1298/2613] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:17 2022] DELETE /identity/v3/projects/999c945890d6427a8c9f4b7022b1a4ed => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:18.026386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.026870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.026980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.046508 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:18.047716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:18.104830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1316/2614] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:18.113601 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.113889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.114168 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.134783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:18.135997 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:18.144831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1299/2615] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:18.149044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.159536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b1e83-7f57-4e18-ad24-28bd3f7df23e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.164155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.164751 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.165028 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.165297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.165986 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:18.166312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.167951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.179383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1317/2616] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:18.183554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.193061 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f45d6-6bdb-4840-a278-5256c78778e2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.197753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.198346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.198627 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.198899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.199515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.201439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.229046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1300/2617] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:18.233858 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.243980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cde5bc-de78-463c-b9cc-0a8ab82a12f3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.248900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:18.301746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.302363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.302644 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.302917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.303536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.305496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.326426 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1318/2618] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:18.331590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.342405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d732c5-45e1-4cdd-bc4f-45afd2217a9e admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.347775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.348372 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.348663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.348937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.349593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.351209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.365860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1301/2619] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:54:18.367450 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.377487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17d438-f525-4b7c-8c2a-20bc46969d0e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.382126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81080849-76ab-4ed9-bead-6dca36edd3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.382798 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81080849-76ab-4ed9-bead-6dca36edd3ec admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.383126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81080849-76ab-4ed9-bead-6dca36edd3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.383289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81080849-76ab-4ed9-bead-6dca36edd3ec admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.383923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81080849-76ab-4ed9-bead-6dca36edd3ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.385506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81080849-76ab-4ed9-bead-6dca36edd3ec admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.396855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1319/2620] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:54:18.401480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.412316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595980f-3474-4f64-9b61-ec985703c5b2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.417045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.417737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.418018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.418294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] PATH_INFO: `/v3/projects/84d1cdee0c3e4a698fe26030b817db63/users/e7c3556ee2df446e8a2b0bba3ce7c609/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.433295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84d1cdee0c3e4a698fe26030b817db63, user_id=e7c3556ee2df446e8a2b0bba3ce7c609, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.435994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.449739 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1302/2621] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:18 2022] PUT /identity/v3/projects/84d1cdee0c3e4a698fe26030b817db63/users/e7c3556ee2df446e8a2b0bba3ce7c609/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:18.461697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.462115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.462449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.490907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] MFA Rules not processed for user `e7c3556ee2df446e8a2b0bba3ce7c609`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:18.492147 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:18.563944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1320/2622] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:18.569069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.634104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eab6eb2-7341-4c68-8d8a-302514f587c4 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.638817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.639721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.640007 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.640276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.640922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.643463 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.674574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1303/2623] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:18.675819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.687025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba241ec-ca25-4216-a587-b0d804e87e23 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.747479 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.751668 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.752580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.752951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.753718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.755742 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.777980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1321/2624] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:18.781980 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.792284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71327a36-03cd-403b-9de3-904fb3c96418 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.796825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.797418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.797735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.798009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.798643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.800304 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.810636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1304/2625] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:18.815127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.824956 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c462ba-426e-466b-90c3-4297154ee7ee admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.829459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.830712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.830990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.831258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] PATH_INFO: `/v3/projects/a92c1040957e4682be6c4c05676eb828/users/8c8b9794c1a342308a32729f4fbd9fe0/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.833545 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:18.847319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a92c1040957e4682be6c4c05676eb828, user_id=8c8b9794c1a342308a32729f4fbd9fe0, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.849995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.863296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1322/2626] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:18 2022] PUT /identity/v3/projects/a92c1040957e4682be6c4c05676eb828/users/8c8b9794c1a342308a32729f4fbd9fe0/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:54:18.867261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.877954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f7ad3-f6d7-4dac-ba44-df2847145a84 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.899799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:18.938036 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f873a8-37bb-44f5-ba2e-4104403d37ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.938643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f873a8-37bb-44f5-ba2e-4104403d37ee admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.938949 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f873a8-37bb-44f5-ba2e-4104403d37ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.939228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f873a8-37bb-44f5-ba2e-4104403d37ee admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.940560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f873a8-37bb-44f5-ba2e-4104403d37ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.942222 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f873a8-37bb-44f5-ba2e-4104403d37ee admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.951889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1305/2627] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:18.956076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:18.966433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c82fd-78e4-400a-a70e-c5a8eef3c77c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:18.970998 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:18.971637 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.971909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.972174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] PATH_INFO: `/v3/projects/a92c1040957e4682be6c4c05676eb828/users/8c8b9794c1a342308a32729f4fbd9fe0/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:18.974097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a92c1040957e4682be6c4c05676eb828, user_id=8c8b9794c1a342308a32729f4fbd9fe0, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:18.976705 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:18.987347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1323/2628] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:18 2022] PUT /identity/v3/projects/a92c1040957e4682be6c4c05676eb828/users/8c8b9794c1a342308a32729f4fbd9fe0/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:18.997427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:18.997798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:18.998080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:19.025754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] MFA Rules not processed for user `8c8b9794c1a342308a32729f4fbd9fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:19.026968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:19.095577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1306/2629] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:18 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:19.106220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:19.108212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:19.186433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a13b01-8363-42b5-8b63-fafc5ba7a94c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:19.190737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85424bf5-0233-4619-b17b-7c36b2242715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:19.191339 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85424bf5-0233-4619-b17b-7c36b2242715 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:19.191616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85424bf5-0233-4619-b17b-7c36b2242715 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:19.191886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85424bf5-0233-4619-b17b-7c36b2242715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:19.200678 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85424bf5-0233-4619-b17b-7c36b2242715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:19.202129 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85424bf5-0233-4619-b17b-7c36b2242715 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:19.211364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1324/2630] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:19 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:19.347223 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:19.410577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d0b793-5bca-4c41-bd90-e23e02152794 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:19.415017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:19.415632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:19.415906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:19.416174 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] PATH_INFO: `/v3/users/e7c3556ee2df446e8a2b0bba3ce7c609` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:19.417327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7c3556ee2df446e8a2b0bba3ce7c609)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:19.419260 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:19.464587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7c3556ee2df446e8a2b0bba3ce7c609'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:19.465238 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7c3556ee2df446e8a2b0bba3ce7c609'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:19.466446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:19.475151 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7c3556ee2df446e8a2b0bba3ce7c609'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:19.475791 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7c3556ee2df446e8a2b0bba3ce7c609'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:19.483381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7c3556ee2df446e8a2b0bba3ce7c609'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:19.500751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1307/2631] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:19 2022] DELETE /identity/v3/users/e7c3556ee2df446e8a2b0bba3ce7c609 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:19.505289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:19.505526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:19.505832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:19.524425 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:19.525667 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:19.583632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1325/2632] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:19.593446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:19.594994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:19.653177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267646f-685f-4042-ad84-3a08d2193c8b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:19.656927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:19.657534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:19.657848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:19.658122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:19.668201 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:19.669787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:19.679166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1308/2633] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:19.955735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:19.957379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:19.971951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291762e-8d11-4fa1-a5d2-46c8ae518d83 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:19.977687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:19.978291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:19.978573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:19.978864 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] PATH_INFO: `/v3/projects/84d1cdee0c3e4a698fe26030b817db63` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:19.979955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] RBAC: Authorizing `identity:delete_project(project_id=84d1cdee0c3e4a698fe26030b817db63)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:19.981909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:20.028391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:20.029804 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84d1cdee0c3e4a698fe26030b817db63', 'request_id': 'req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.037296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84d1cdee0c3e4a698fe26030b817db63', 'request_id': 'req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.044840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84d1cdee0c3e4a698fe26030b817db63', 'request_id': 'req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.056066 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] The token cache is being invalidate because project 84d1cdee0c3e4a698fe26030b817db63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:20.056348 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.056974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.058463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1326/2634] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:19 2022] DELETE /identity/v3/projects/84d1cdee0c3e4a698fe26030b817db63 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:20.062650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:20.072802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f84a0-796d-4e21-b289-57526cb742b7 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:20.132119 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:20.132772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.133054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.133324 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] PATH_INFO: `/v3/users/8c8b9794c1a342308a32729f4fbd9fe0` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.134954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8b9794c1a342308a32729f4fbd9fe0)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:20.136858 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:20.181861 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8b9794c1a342308a32729f4fbd9fe0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.182903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8b9794c1a342308a32729f4fbd9fe0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.190600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8b9794c1a342308a32729f4fbd9fe0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.191237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8b9794c1a342308a32729f4fbd9fe0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.198753 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c290b007-9894-4259-b63a-78a49dfaf2d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8b9794c1a342308a32729f4fbd9fe0'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.204047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:20.218560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1309/2635] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:20 2022] DELETE /identity/v3/users/8c8b9794c1a342308a32729f4fbd9fe0 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:20.506367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:20.508238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:20.533110 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5cd2c7-36b9-4cde-86f2-a21bf67f947a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:20.590738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:20.591373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.591651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.591923 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] PATH_INFO: `/v3/projects/a92c1040957e4682be6c4c05676eb828` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.593044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a92c1040957e4682be6c4c05676eb828)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:20.595004 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:20.642547 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a92c1040957e4682be6c4c05676eb828', 'request_id': 'req-e59da02e-6201-436a-aab4-148b6e5d803b'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.648145 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a92c1040957e4682be6c4c05676eb828', 'request_id': 'req-e59da02e-6201-436a-aab4-148b6e5d803b'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.655594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a92c1040957e4682be6c4c05676eb828', 'request_id': 'req-e59da02e-6201-436a-aab4-148b6e5d803b'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.667897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] The token cache is being invalidate because project a92c1040957e4682be6c4c05676eb828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:20.668191 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.668915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-e59da02e-6201-436a-aab4-148b6e5d803b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:20.670453 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1327/2636] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:20 2022] DELETE /identity/v3/projects/a92c1040957e4682be6c4c05676eb828 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:20.675760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.676048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.676327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.695939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:20.697160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:20.756178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1310/2637] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:20.764034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.764319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.764592 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.784573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:20.786352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:20.794160 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1328/2638] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:20.798404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:20.810406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ac3bf-2bc6-41b4-a933-30c1be84b6e6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:20.815307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:20.815926 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.816207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.816481 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.817141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:20.817445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:20.819143 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:20.831331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1311/2639] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:20.837030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:20.846806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa139e5-12b3-4a37-992a-70142067fe60 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:20.851357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:20.851944 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.852224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.852494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.853131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:20.855134 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:20.882954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1329/2640] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:20 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:54:20.887244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:20.897525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650ed96-67e5-4f40-b5b6-54e24b890679 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:20.954345 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:20.954992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:20.955242 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:20.955510 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:20.956152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:20.958177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:20.980520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1312/2641] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:20 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:20.984857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:20.996011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14908e00-f17a-4f6f-80ea-ff02787f3f3b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.000832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.001434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.001761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.002021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.002657 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.004253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.014720 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1330/2642] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:21.018844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.029352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc46e7e-2be2-49ed-a820-fcdeb741ca0c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.036096 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf65b0-03be-465d-b7f9-ea5f74f49283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.036774 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf65b0-03be-465d-b7f9-ea5f74f49283 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.037101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf65b0-03be-465d-b7f9-ea5f74f49283 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.037393 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf65b0-03be-465d-b7f9-ea5f74f49283 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.038055 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf65b0-03be-465d-b7f9-ea5f74f49283 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.039930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf65b0-03be-465d-b7f9-ea5f74f49283 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.053757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1313/2643] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:21.058017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.068044 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e735a8-ec3a-417b-b58e-61798be6c432 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.073992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.074780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.075151 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.075503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] PATH_INFO: `/v3/projects/c8cda8cfe9934612b9813f33e879dfaa/users/f884e05866cb4468acfeebe5c9d6ed3d/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.093987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8cda8cfe9934612b9813f33e879dfaa, user_id=f884e05866cb4468acfeebe5c9d6ed3d, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.096328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.110929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1331/2644] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:21 2022] PUT /identity/v3/projects/c8cda8cfe9934612b9813f33e879dfaa/users/f884e05866cb4468acfeebe5c9d6ed3d/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:21.121754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.122255 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.122360 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.150656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] MFA Rules not processed for user `f884e05866cb4468acfeebe5c9d6ed3d`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:21.151875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:21.195133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:21.223328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1314/2645] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:21.227885 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.232621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:21.306342 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff311ab-6c34-4cc3-8b0f-b48b12c44d86 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.311671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.312292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.312572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.312984 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.313669 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.315601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.348563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1332/2646] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:21 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:21.352765 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.367127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6363c2-0a8e-455f-9ab7-091034300117 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.430931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.431533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.431809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.432188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.432804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.434925 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.460226 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1315/2647] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:21 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:54:21.461416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.472777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7054e-7878-44df-aff7-98fbb7195bd5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.477408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.478054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.478337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.478732 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.479379 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.480972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.492015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1333/2648] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:21.496219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.506992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d7d41b-ed1e-4bd5-9efa-d4c169a7e89b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.512679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.513352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.513674 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.513953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] PATH_INFO: `/v3/projects/f8cbab64b5334bcf8d8918d12796a618/users/15d25fbe6ccb452788ec4715229b6742/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.531963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cbab64b5334bcf8d8918d12796a618, user_id=15d25fbe6ccb452788ec4715229b6742, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.534883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.549711 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1316/2649] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:21 2022] PUT /identity/v3/projects/f8cbab64b5334bcf8d8918d12796a618/users/15d25fbe6ccb452788ec4715229b6742/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:21.553212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.563432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba26308-6f45-426f-a363-021fe69d8557 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.621680 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6df4c-cb45-4da0-baaa-2c7e002c4b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.622200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6df4c-cb45-4da0-baaa-2c7e002c4b69 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.622476 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6df4c-cb45-4da0-baaa-2c7e002c4b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.622741 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6df4c-cb45-4da0-baaa-2c7e002c4b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.623372 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6df4c-cb45-4da0-baaa-2c7e002c4b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.624952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6df4c-cb45-4da0-baaa-2c7e002c4b69 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.635748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1334/2650] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:21.640114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.651400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb175a4-0399-4d8c-9904-e3cc0f986b7c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.655913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.656563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.656837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.657106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] PATH_INFO: `/v3/projects/f8cbab64b5334bcf8d8918d12796a618/users/15d25fbe6ccb452788ec4715229b6742/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.659091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8cbab64b5334bcf8d8918d12796a618, user_id=15d25fbe6ccb452788ec4715229b6742, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.661748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.672568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1317/2651] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:21 2022] PUT /identity/v3/projects/f8cbab64b5334bcf8d8918d12796a618/users/15d25fbe6ccb452788ec4715229b6742/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:21.682976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.683261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.683536 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.714033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] MFA Rules not processed for user `15d25fbe6ccb452788ec4715229b6742`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:21.715791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:21.783565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:21.789003 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1335/2652] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:21.801493 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.803273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:21.863466 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122f5936-865e-4a70-8ae9-d4c3b7f82d34 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.867621 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.868232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.868512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.868789 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:21.878607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:21.879974 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:21.889374 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1318/2653] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:21.961903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:21.971318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf261b2-a469-4180-bbad-fd268e26e591 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:21.974865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2caf5e91-2ab9-4397-8285-d4b62abffff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:21.975454 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf5e91-2ab9-4397-8285-d4b62abffff8 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:21.975727 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf5e91-2ab9-4397-8285-d4b62abffff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:21.975994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2caf5e91-2ab9-4397-8285-d4b62abffff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:22.032142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf5e91-2ab9-4397-8285-d4b62abffff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:22.033671 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2caf5e91-2ab9-4397-8285-d4b62abffff8 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:22.042623 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1336/2654] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:22.311623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:22.352340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:22.376136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac33dc8-e278-4ac6-8ef8-9ce8c6185b5f service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:22.380932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:22.381554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:22.381860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:22.382130 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] PATH_INFO: `/v3/users/f884e05866cb4468acfeebe5c9d6ed3d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:22.383284 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f884e05866cb4468acfeebe5c9d6ed3d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:22.385163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:22.432190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f884e05866cb4468acfeebe5c9d6ed3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:22.432844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f884e05866cb4468acfeebe5c9d6ed3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:22.439846 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f884e05866cb4468acfeebe5c9d6ed3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:22.440537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f884e05866cb4468acfeebe5c9d6ed3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:22.448443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f884e05866cb4468acfeebe5c9d6ed3d'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:22.466722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1319/2655] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:22 2022] DELETE /identity/v3/users/f884e05866cb4468acfeebe5c9d6ed3d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:22.471677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:22.471967 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:22.472245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:22.492872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:22.494126 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:22.555602 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1337/2656] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:22.565158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:22.566817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:22.628988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5647120-a682-4bfd-b818-dc3524416a76 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:22.632641 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:22.633248 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:22.636827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:22.636827 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:22.643738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:22.645179 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:22.654844 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1320/2657] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:22.923013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:22.924458 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:22.944027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd1ad53-88ea-43b3-a568-0fcb8305cf42 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:22.949159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:22.949829 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:22.950111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:22.950383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] PATH_INFO: `/v3/projects/c8cda8cfe9934612b9813f33e879dfaa` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:22.951503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8cda8cfe9934612b9813f33e879dfaa)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:22.953415 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.000620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8cda8cfe9934612b9813f33e879dfaa', 'request_id': 'req-d552f58e-a1c8-407b-bd98-6df7815610fa'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.006601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8cda8cfe9934612b9813f33e879dfaa', 'request_id': 'req-d552f58e-a1c8-407b-bd98-6df7815610fa'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.014076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8cda8cfe9934612b9813f33e879dfaa', 'request_id': 'req-d552f58e-a1c8-407b-bd98-6df7815610fa'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.025591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] The token cache is being invalidate because project c8cda8cfe9934612b9813f33e879dfaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:23.025877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.026501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.027955 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1338/2658] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:22 2022] DELETE /identity/v3/projects/c8cda8cfe9934612b9813f33e879dfaa => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:23.032050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.043228 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea24a2f3-b1ea-4196-bc49-99317c43255e service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.099051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.099703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.099964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.100236 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] PATH_INFO: `/v3/users/15d25fbe6ccb452788ec4715229b6742` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.101356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=15d25fbe6ccb452788ec4715229b6742)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.103317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.104792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:23.152737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15d25fbe6ccb452788ec4715229b6742'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.153320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15d25fbe6ccb452788ec4715229b6742'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.160142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15d25fbe6ccb452788ec4715229b6742'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.160777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15d25fbe6ccb452788ec4715229b6742'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.168140 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e8d70b9-9232-4b45-b60f-adea00fd13cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15d25fbe6ccb452788ec4715229b6742'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.184971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1321/2659] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:23 2022] DELETE /identity/v3/users/15d25fbe6ccb452788ec4715229b6742 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:23.451109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.452518 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:23.468197 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d552f58e-a1c8-407b-bd98-6df7815610fa admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.486953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:23.528647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.529337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.529656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.529935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] PATH_INFO: `/v3/projects/f8cbab64b5334bcf8d8918d12796a618` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.531065 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8cbab64b5334bcf8d8918d12796a618)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.533005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.581787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8cbab64b5334bcf8d8918d12796a618', 'request_id': 'req-921bcd81-821a-43ce-985f-8455141c5784'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.587628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8cbab64b5334bcf8d8918d12796a618', 'request_id': 'req-921bcd81-821a-43ce-985f-8455141c5784'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.596337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8cbab64b5334bcf8d8918d12796a618', 'request_id': 'req-921bcd81-821a-43ce-985f-8455141c5784'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.608396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] The token cache is being invalidate because project f8cbab64b5334bcf8d8918d12796a618 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:23.608684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.609344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-921bcd81-821a-43ce-985f-8455141c5784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:23.610835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1339/2660] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:23 2022] DELETE /identity/v3/projects/f8cbab64b5334bcf8d8918d12796a618 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:23.616171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.616647 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.616744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.637077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:23.638325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:23.702341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1322/2661] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:23.711046 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.711327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.711594 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.730932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:23.732132 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:23.740737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1340/2662] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:23.745042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.755378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba86e6-dd3a-4e8f-ba2a-4c006feb4d86 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.759884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.760470 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.760750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.761018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.761684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:23.761984 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.763566 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.775167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1323/2663] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:23.779296 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.789171 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f9916-d5ae-4baa-8341-3f7bf07f8895 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.794597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.795227 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.795510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.795783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.796414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.798424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.827356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1341/2664] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:23 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:54:23.831476 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.841542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a90a2-9530-43a6-9ce2-a881dc312c35 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.899314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.899915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.900197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.900475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.901094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.903068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.913918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:23.929870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1324/2665] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:23 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:23.931048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.942207 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5e17ac-a355-42a8-ab31-a6cbfdac4dd9 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.947246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.947848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.948131 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.948419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.949054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.950664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.961217 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1342/2666] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:54:23.965399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:23.975341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c032415f-1f4c-4911-b0f9-8e418eb7faa2 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:23.979924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb845aa8-53a9-4267-84fc-ab97a16c3b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:23.980523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845aa8-53a9-4267-84fc-ab97a16c3b59 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:23.980802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845aa8-53a9-4267-84fc-ab97a16c3b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:23.981076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb845aa8-53a9-4267-84fc-ab97a16c3b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:23.981744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb845aa8-53a9-4267-84fc-ab97a16c3b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:23.983596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb845aa8-53a9-4267-84fc-ab97a16c3b59 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:23.994106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1325/2667] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:23.998150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:24.008341 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d02c49-c28a-4aa9-bf88-0587eb5c1f26 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:24.013006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:24.013684 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.013964 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.014237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] PATH_INFO: `/v3/projects/53226af9e8d2478d96135e433857e5b9/users/dd239a9a92e7439c9792d205efaff3d6/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.029225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] RBAC: Authorizing `identity:create_grant(project_id=53226af9e8d2478d96135e433857e5b9, user_id=dd239a9a92e7439c9792d205efaff3d6, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:24.031881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:24.045244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1343/2668] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:23 2022] PUT /identity/v3/projects/53226af9e8d2478d96135e433857e5b9/users/dd239a9a92e7439c9792d205efaff3d6/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:24.055352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.055661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.055940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.084799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] MFA Rules not processed for user `dd239a9a92e7439c9792d205efaff3d6`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:24.086056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.151700 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1326/2669] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:24.162761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:24.164451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.216111 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:24.223918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7fff93-2b3b-436d-82e4-fff95a03090c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:24.228027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0535d62-effe-45a4-bdb3-bd1611b53443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:24.228661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535d62-effe-45a4-bdb3-bd1611b53443 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.228935 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535d62-effe-45a4-bdb3-bd1611b53443 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.229202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0535d62-effe-45a4-bdb3-bd1611b53443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.238550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0535d62-effe-45a4-bdb3-bd1611b53443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:24.240088 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0535d62-effe-45a4-bdb3-bd1611b53443 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:24.249201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1344/2670] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:24.270899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:24.333077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d84fc2-1ae6-491d-9260-e7110a08866d None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:24.337658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:24.338291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.338569 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.338841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] PATH_INFO: `/v3/users/dd239a9a92e7439c9792d205efaff3d6` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.339966 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd239a9a92e7439c9792d205efaff3d6)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:24.342013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:24.388077 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd239a9a92e7439c9792d205efaff3d6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.388735 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd239a9a92e7439c9792d205efaff3d6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.397265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd239a9a92e7439c9792d205efaff3d6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.397932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd239a9a92e7439c9792d205efaff3d6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.405282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd239a9a92e7439c9792d205efaff3d6'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.416148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:24.424693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1327/2671] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:24 2022] DELETE /identity/v3/users/dd239a9a92e7439c9792d205efaff3d6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:24.429330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.429666 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.429938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.448445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:24.449662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.507789 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1345/2672] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:24.517290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:24.519356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.574968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb3872-3c2a-4c6f-908e-e24c37267b1a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:24.578536 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be08502-6521-40e6-8af0-b1d86e4d1061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:24.579135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be08502-6521-40e6-8af0-b1d86e4d1061 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.579410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be08502-6521-40e6-8af0-b1d86e4d1061 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.579678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be08502-6521-40e6-8af0-b1d86e4d1061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.588998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be08502-6521-40e6-8af0-b1d86e4d1061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:24.590435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be08502-6521-40e6-8af0-b1d86e4d1061 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:24.600614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1328/2673] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:24.867381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:24.868834 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:24.881183 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f380b-0e09-47ec-bd67-13e140fd7159 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:24.885990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:24.886620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.886895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.887161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] PATH_INFO: `/v3/projects/53226af9e8d2478d96135e433857e5b9` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.888279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53226af9e8d2478d96135e433857e5b9)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:24.890208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:24.937033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53226af9e8d2478d96135e433857e5b9', 'request_id': 'req-501efd19-1804-47a6-95bb-20c57aabfe2f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.943314 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53226af9e8d2478d96135e433857e5b9', 'request_id': 'req-501efd19-1804-47a6-95bb-20c57aabfe2f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.950514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53226af9e8d2478d96135e433857e5b9', 'request_id': 'req-501efd19-1804-47a6-95bb-20c57aabfe2f'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.961257 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] The token cache is being invalidate because project 53226af9e8d2478d96135e433857e5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:24.961709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.962337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-501efd19-1804-47a6-95bb-20c57aabfe2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:24.963802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1346/2674] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:24 2022] DELETE /identity/v3/projects/53226af9e8d2478d96135e433857e5b9 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:24.968731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:24.969230 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:24.969341 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:24.989233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:24.990458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:25.047832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1329/2675] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:25.055975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.056266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.056540 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.077021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:25.078238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:25.085928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1347/2676] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:25.090172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.100001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0acc06d-b80a-47ec-beb1-00a12a09957a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.104666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.105281 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.105617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.105893 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.106535 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:25.106865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.108750 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.121097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1330/2677] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 05:54:25.125176 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.135272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c62dd7-3eb2-4c7c-9f06-da3799fadba9 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.143186 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.143781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.144058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.144325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.144958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.146877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.175381 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1348/2678] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:25 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:25.179522 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.190056 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c1f421-e946-4e57-a406-d36bfd696efc admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.245086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.245724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.246024 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.246273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.246900 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.248821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.270549 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1331/2679] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:25 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:25.275487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.285499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf8c72-0ea6-4895-9139-ba32633a1094 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.290177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.290774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.291055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.291328 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.291966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.293624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.304167 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1349/2680] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:25.308199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.318552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac055b4-4c3f-480c-bdb8-bebc7523a859 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.323210 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d683957-961b-4f22-aca1-d7ee93aac771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.323803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683957-961b-4f22-aca1-d7ee93aac771 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.324089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683957-961b-4f22-aca1-d7ee93aac771 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.324364 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d683957-961b-4f22-aca1-d7ee93aac771 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.325004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683957-961b-4f22-aca1-d7ee93aac771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.326663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d683957-961b-4f22-aca1-d7ee93aac771 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.337293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1332/2681] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:54:25.341534 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.351925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ade7b-9768-4f35-bc28-b18732a7a897 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.356687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.357369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.357697 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.358234 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] PATH_INFO: `/v3/projects/2975c35e70124ec0bcc6168915f2066c/users/7ca3de86ba6a44fbb692a2477bfef02a/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.373245 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2975c35e70124ec0bcc6168915f2066c, user_id=7ca3de86ba6a44fbb692a2477bfef02a, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.375947 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.388865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1350/2682] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:25 2022] PUT /identity/v3/projects/2975c35e70124ec0bcc6168915f2066c/users/7ca3de86ba6a44fbb692a2477bfef02a/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:25.453351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.453666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.453944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.461107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:25.484826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] MFA Rules not processed for user `7ca3de86ba6a44fbb692a2477bfef02a`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:25.486696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:25.549553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1333/2683] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:25.560827 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:25.562505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:25.573253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:25.622389 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b775ee03-df07-4a9e-9d2f-95f0c45c6549 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:25.626000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '960468d880f34ea2a267b6ada5fb30bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:25.626628 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:25.626908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:25.627179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:25.635352 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:25.636776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:25.645972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1351/2684] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:35.546083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=31 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:35.641514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=22 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:36.169419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:36.170875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:36.226804 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f9e19-1b86-4724-8370-6e2c13f08943 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:36.230406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:36.231004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:36.231282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:36.231550 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:36.233499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:36.293257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:36.294721 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:36.304120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1334/2685] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:54:40.432030 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:40.433496 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:40.530460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f41e61-61db-4122-9358-b1cc00128f10 service glance] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:40.535691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:40.536297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:40.536572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:40.536898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:40.538676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:40.551261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:40.552838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:40.563951 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1352/2686] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:42.148513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:42.149982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:42.161285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff105ad4-27c9-412a-a8af-bad3f758e782 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:42.165167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:42.165806 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:42.166133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:42.166357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:42.168038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:42.198261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:42.235908 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:42.238129 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54ed80-f958-4d76-a049-473e2fbd3c6e service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:42.248459 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1335/2687] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:42.263843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7f34da-0906-4643-8904-1499493f47aa service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:42.268820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:42.269522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:42.269764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:42.270043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] PATH_INFO: `/v3/users/7ca3de86ba6a44fbb692a2477bfef02a` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:42.271985 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca3de86ba6a44fbb692a2477bfef02a)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:42.273180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:42.328356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca3de86ba6a44fbb692a2477bfef02a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.329338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca3de86ba6a44fbb692a2477bfef02a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.338103 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca3de86ba6a44fbb692a2477bfef02a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.338928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca3de86ba6a44fbb692a2477bfef02a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.353253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca3de86ba6a44fbb692a2477bfef02a'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.354713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:42.373398 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1353/2688] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:42 2022] DELETE /identity/v3/users/7ca3de86ba6a44fbb692a2477bfef02a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:42.378084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:42.379082 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:42.379243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:42.399128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:42.400344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:42.447107 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=100 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:42.461793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1336/2689] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:42.471395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:42.473395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:42.532333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7811059-7367-40e3-84c9-99bb49cff123 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:42.536957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fee12-0e59-45aa-9def-82a632d64f5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:42.537588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fee12-0e59-45aa-9def-82a632d64f5a service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:42.537874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fee12-0e59-45aa-9def-82a632d64f5a service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:42.538148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fee12-0e59-45aa-9def-82a632d64f5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:42.547411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fee12-0e59-45aa-9def-82a632d64f5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:42.548844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fee12-0e59-45aa-9def-82a632d64f5a service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:42.559568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1354/2690] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:42.843030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:42.844620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:42.859405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52b76f-2818-454e-8eac-d01822e4e747 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:42.864579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:42.865240 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:42.865519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:42.865833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] PATH_INFO: `/v3/projects/2975c35e70124ec0bcc6168915f2066c` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:42.867011 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=2975c35e70124ec0bcc6168915f2066c)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:42.868985 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:42.921385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2975c35e70124ec0bcc6168915f2066c', 'request_id': 'req-c785e643-ac3c-450c-83f9-63de19fc49ed'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.927625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2975c35e70124ec0bcc6168915f2066c', 'request_id': 'req-c785e643-ac3c-450c-83f9-63de19fc49ed'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.934758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2975c35e70124ec0bcc6168915f2066c', 'request_id': 'req-c785e643-ac3c-450c-83f9-63de19fc49ed'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.948405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] The token cache is being invalidate because project 2975c35e70124ec0bcc6168915f2066c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:42.948698 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.949353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c785e643-ac3c-450c-83f9-63de19fc49ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:42.950888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1337/2691] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:42 2022] DELETE /identity/v3/projects/2975c35e70124ec0bcc6168915f2066c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:54:42.955848 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:42.956612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:42.956884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:42.977429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:42.978780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:43.038929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1355/2692] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:43.046617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.046902 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.047177 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.067199 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:43.068004 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:43.076173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1338/2693] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:43.080431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.090540 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84700589-bc92-407f-bf01-bed2e49afb55 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.095025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.095621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.095897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.096283 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.097473 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:43.097816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.099388 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.111164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1356/2694] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:43.115458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.124878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa0f0-0855-4251-9bf0-7d7302b1d081 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.129717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.130320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.130598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.130869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.131501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.133399 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.163197 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1339/2695] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:43 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:43.167404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.177662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3f76c-6148-424b-a693-16ec40e3cea2 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.239511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.240133 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.240412 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.240686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.241324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.243306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.265687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1357/2696] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:43 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:43.270561 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.282587 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188a52b-5f79-4341-afb5-5dc70f4f0c26 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.287404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.288009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.288310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.288585 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.289305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.290952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.301340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1340/2697] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:43.305873 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.316650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4c867-084f-4ae8-be47-5b58ade21550 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.321689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460230ad-13e5-4727-bff8-c77fba861724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.322298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460230ad-13e5-4727-bff8-c77fba861724 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.322583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460230ad-13e5-4727-bff8-c77fba861724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.322857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460230ad-13e5-4727-bff8-c77fba861724 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.323499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460230ad-13e5-4727-bff8-c77fba861724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.325106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460230ad-13e5-4727-bff8-c77fba861724 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.327298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:43.338102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1358/2698] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:43.342105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.352616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f547bbf-33f1-48c5-8e16-0420ad99b2db admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.358185 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.358854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.359136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.359435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] PATH_INFO: `/v3/projects/5bcab6f874b54f9d89052636330ba1e4/users/c9ed877ebf614ab5bf1a2eeb6fad4513/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.375296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bcab6f874b54f9d89052636330ba1e4, user_id=c9ed877ebf614ab5bf1a2eeb6fad4513, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.378571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.396050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1341/2699] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:43 2022] PUT /identity/v3/projects/5bcab6f874b54f9d89052636330ba1e4/users/c9ed877ebf614ab5bf1a2eeb6fad4513/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:43.407606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.408016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.408394 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.443210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] MFA Rules not processed for user `c9ed877ebf614ab5bf1a2eeb6fad4513`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:43.444872 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:43.516517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1359/2700] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:43.529677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:43.531325 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:43.604068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f830a29-a781-46aa-841d-b03715814e16 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:43.607716 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ffcc4-6e25-4e01-8667-7a776f9b2ab8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:43.608579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ffcc4-6e25-4e01-8667-7a776f9b2ab8 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:43.608857 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ffcc4-6e25-4e01-8667-7a776f9b2ab8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:43.609128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ffcc4-6e25-4e01-8667-7a776f9b2ab8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:43.617478 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ffcc4-6e25-4e01-8667-7a776f9b2ab8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:43.618944 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ffcc4-6e25-4e01-8667-7a776f9b2ab8 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:43.627760 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1342/2701] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:46.288272 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:46.357862 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cf8db-6b87-49f1-905b-bc019d069bec None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:46.363299 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:46.364445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:46.364844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:46.365168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] PATH_INFO: `/v3/users/c9ed877ebf614ab5bf1a2eeb6fad4513` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:46.366621 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ed877ebf614ab5bf1a2eeb6fad4513)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:46.368823 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:46.427274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ed877ebf614ab5bf1a2eeb6fad4513'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:46.428250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ed877ebf614ab5bf1a2eeb6fad4513'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:46.438202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ed877ebf614ab5bf1a2eeb6fad4513'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:46.438869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ed877ebf614ab5bf1a2eeb6fad4513'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:46.453208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ed877ebf614ab5bf1a2eeb6fad4513'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:46.470306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1360/2702] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:46 2022] DELETE /identity/v3/users/c9ed877ebf614ab5bf1a2eeb6fad4513 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:46.475044 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:46.475558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:46.475851 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:46.496327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:46.497915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:46.504136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:46.569879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1343/2703] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:46.579639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:46.581811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:46.591874 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:46.645252 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec303c35-7db3-4ff2-b552-e2e6e136c670 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:46.648840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe860480-efc3-441d-aca3-ccd30d6aa2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:46.649456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860480-efc3-441d-aca3-ccd30d6aa2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:46.649792 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860480-efc3-441d-aca3-ccd30d6aa2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:46.650069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe860480-efc3-441d-aca3-ccd30d6aa2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:46.659681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860480-efc3-441d-aca3-ccd30d6aa2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:46.661115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe860480-efc3-441d-aca3-ccd30d6aa2a2 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:46.671768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1361/2704] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:46.961025 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:46.963249 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:46.985383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4004e7a-f65e-4991-af50-e5adfb39f1f6 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.004162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.005232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.006378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.006378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] PATH_INFO: `/v3/projects/5bcab6f874b54f9d89052636330ba1e4` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.007294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bcab6f874b54f9d89052636330ba1e4)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.010188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.072959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bcab6f874b54f9d89052636330ba1e4', 'request_id': 'req-9b91402f-9238-48c1-9f3e-9c46d96d36b4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:47.079486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bcab6f874b54f9d89052636330ba1e4', 'request_id': 'req-9b91402f-9238-48c1-9f3e-9c46d96d36b4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:47.087295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bcab6f874b54f9d89052636330ba1e4', 'request_id': 'req-9b91402f-9238-48c1-9f3e-9c46d96d36b4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:47.101767 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] The token cache is being invalidate because project 5bcab6f874b54f9d89052636330ba1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:47.102344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:47.103086 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-9b91402f-9238-48c1-9f3e-9c46d96d36b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:47.105054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1344/2705] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:46 2022] DELETE /identity/v3/projects/5bcab6f874b54f9d89052636330ba1e4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:47.110035 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.111115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.111115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.136499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:47.137774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:47.215093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1362/2706] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:47.223060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.223349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.223630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.246899 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:47.248167 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:47.262684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1345/2707] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:47.271048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.282795 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e453f-edd3-4607-9b32-209686f89399 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.287624 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.288237 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.288654 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.289192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.289610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:47.289918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.291726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.307127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1363/2708] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:47.308818 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.319109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bde564-9bfb-4e73-9916-b1351036213a None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.323941 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.324479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.324754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.325022 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.325690 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.327586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.361047 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1346/2709] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:47 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:47.365785 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.376677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7a030e-9a53-4f9f-a3c7-0287e17653ba admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.444153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.445121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.445408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.445871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.446965 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.449138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.472641 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1364/2710] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:47 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:47.476860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.489319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1436f-4a27-49fa-a363-2e4607151f63 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.499733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.500490 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.500831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.501170 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.501930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.504244 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.517423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1347/2711] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:47.521769 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.533150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9308349-a6da-4949-a6e8-46e6325347aa admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.537971 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93287783-e49b-4bd4-9e11-3aa03be080e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.538576 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287783-e49b-4bd4-9e11-3aa03be080e6 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.538855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287783-e49b-4bd4-9e11-3aa03be080e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.539127 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93287783-e49b-4bd4-9e11-3aa03be080e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.539760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287783-e49b-4bd4-9e11-3aa03be080e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.541350 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93287783-e49b-4bd4-9e11-3aa03be080e6 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.553636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1365/2712] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:47.557879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.568653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcadbd4-4870-45fe-8dbd-4c42db657123 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.573275 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.574002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.574233 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.574507 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] PATH_INFO: `/v3/projects/02260c0eb59e42a581e20db9d0f12f21/users/4737f56d9d874539b1468870b0903a5e/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.590655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02260c0eb59e42a581e20db9d0f12f21, user_id=4737f56d9d874539b1468870b0903a5e, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.593343 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.606703 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1348/2713] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:47 2022] PUT /identity/v3/projects/02260c0eb59e42a581e20db9d0f12f21/users/4737f56d9d874539b1468870b0903a5e/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:47.617898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.618264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.618569 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.646256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] MFA Rules not processed for user `4737f56d9d874539b1468870b0903a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:47.647488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:47.651555 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:47.720060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1366/2714] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:47 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:47.732542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:47.734821 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:47.773810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:47.802421 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af1bcf-e57c-4b1b-83f1-042db27390f7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:47.807221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:47.807950 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:47.808302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:47.808643 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:47.818451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:47.819957 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:47.830458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1349/2715] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:47 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:48.727533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:48.731509 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:48.803484 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:48.806597 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:48.825546 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b910491-4f65-4991-a781-5e5e0bee521b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:48.835457 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:48.836137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:48.836370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:48.836826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:48.838928 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:48.852646 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:48.854417 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:48.865363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1367/2716] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:48.897873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22512ade-d133-4a79-9e45-775c8bc957b5 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:48.902259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:48.902910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:48.903169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:48.903439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] PATH_INFO: `/v3/users/135e823ff7f64beab0689c3a84cc268f` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:48.904560 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=135e823ff7f64beab0689c3a84cc268f)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:48.906542 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:48.963810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135e823ff7f64beab0689c3a84cc268f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:48.964465 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135e823ff7f64beab0689c3a84cc268f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:48.973787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135e823ff7f64beab0689c3a84cc268f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:48.974487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135e823ff7f64beab0689c3a84cc268f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:48.983331 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135e823ff7f64beab0689c3a84cc268f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.002391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1350/2717] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:48 2022] DELETE /identity/v3/users/135e823ff7f64beab0689c3a84cc268f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:54:49.177707 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:49.191246 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc24fe8-4ede-4c6b-a084-7c7321e97571 service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:49.250249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:49.250878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:49.251158 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:49.251429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] PATH_INFO: `/v3/projects/1406fa515f224f4b8507331fd22ea4fd` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:49.252526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1406fa515f224f4b8507331fd22ea4fd)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:49.255008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:49.295255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:49.305439 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1406fa515f224f4b8507331fd22ea4fd', 'request_id': 'req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.311034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1406fa515f224f4b8507331fd22ea4fd', 'request_id': 'req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.319516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1406fa515f224f4b8507331fd22ea4fd', 'request_id': 'req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.331995 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] The token cache is being invalidate because project 1406fa515f224f4b8507331fd22ea4fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:49.332356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.332952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.334456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1368/2718] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:49 2022] DELETE /identity/v3/projects/1406fa515f224f4b8507331fd22ea4fd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:49.338446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:49.339879 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:49.353195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d7a9c-1efc-4831-8831-5adc62ac0bc7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:49.404083 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:49.414727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:49.415402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:49.415919 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:49.416454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] PATH_INFO: `/v3/users/b982664b971045f78b1a3813fb23080c` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:49.417918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b982664b971045f78b1a3813fb23080c)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:49.419839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:49.471677 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b982664b971045f78b1a3813fb23080c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.472340 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b982664b971045f78b1a3813fb23080c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.479845 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b982664b971045f78b1a3813fb23080c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.480494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b982664b971045f78b1a3813fb23080c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.487921 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-e48dead3-a89c-4bd8-bec6-2992b15101dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b982664b971045f78b1a3813fb23080c'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.513623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1351/2719] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:49 2022] DELETE /identity/v3/users/b982664b971045f78b1a3813fb23080c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:49.692434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:49.693957 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:49.706632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1da9b-9cb8-4cbf-945f-d4780b46d14c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:49.764155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:49.764784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:49.765048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:49.765319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] PATH_INFO: `/v3/projects/5332eaf50eaf416daddbfc142e3917a8` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:49.766464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5332eaf50eaf416daddbfc142e3917a8)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:49.768376 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:49.813192 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5332eaf50eaf416daddbfc142e3917a8', 'request_id': 'req-90581244-7dfb-4591-a02e-7f25cff7a668'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.819411 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5332eaf50eaf416daddbfc142e3917a8', 'request_id': 'req-90581244-7dfb-4591-a02e-7f25cff7a668'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.826587 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5332eaf50eaf416daddbfc142e3917a8', 'request_id': 'req-90581244-7dfb-4591-a02e-7f25cff7a668'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.837465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] The token cache is being invalidate because project 5332eaf50eaf416daddbfc142e3917a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:49.837779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.838423 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-90581244-7dfb-4591-a02e-7f25cff7a668 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:49.839893 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1369/2720] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:49 2022] DELETE /identity/v3/projects/5332eaf50eaf416daddbfc142e3917a8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:54:49.844802 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:49.845266 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:49.845363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:49.865400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:49.866623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:49.926576 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1352/2721] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:49.935203 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:49.935488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:49.935760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:49.955560 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:49.956717 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:49.964492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1370/2722] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:49.968640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:49.979564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6453b0f4-69a9-47f0-9c50-43f6cc1f60cf None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:49.984448 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:49.985071 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:49.985350 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:49.985661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:49.986307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:54:49.986608 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:49.988830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.000319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1353/2723] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:54:50.004559 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:50.013733 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57f17-fed6-4dc6-bea1-31899fdaf841 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:50.018516 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:50.019109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.019387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.019660 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.020280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:50.022221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.050830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1371/2724] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:50 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:50.054913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:50.064968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddb831c-dde7-49f4-82dd-b539c05b1b9c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:50.121538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:50.122173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.122455 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.122727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.123352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:50.125264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.133075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:50.148365 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1354/2725] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:50 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:50.153029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:50.165269 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9341d51a-0adf-45c2-b10f-6969a2a974e7 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:50.170568 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:50.171156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.171456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.171721 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.172384 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:50.174005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.184395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1372/2726] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:50.188323 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:50.198555 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bb4bb-4d58-4428-a73a-8ddf42bc16ab admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:50.203190 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b0b85-a47d-4ce5-b57b-5e86a31234c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:50.203788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b0b85-a47d-4ce5-b57b-5e86a31234c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.204068 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b0b85-a47d-4ce5-b57b-5e86a31234c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.204338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b0b85-a47d-4ce5-b57b-5e86a31234c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.205021 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b0b85-a47d-4ce5-b57b-5e86a31234c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:50.206699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b0b85-a47d-4ce5-b57b-5e86a31234c9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.217349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1355/2727] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:50.221293 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:50.232137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cb338-0a81-49bb-b7fb-02de09811dbd admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:50.237043 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:50.238360 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.238636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.238905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] PATH_INFO: `/v3/projects/76656ddf3c744ba1a32cfaa37e278ef2/users/389ae44e6cfd4f5fbe56063e5b0ae787/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.250606 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:50.254107 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76656ddf3c744ba1a32cfaa37e278ef2, user_id=389ae44e6cfd4f5fbe56063e5b0ae787, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:50.256897 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.271383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1373/2728] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:50 2022] PUT /identity/v3/projects/76656ddf3c744ba1a32cfaa37e278ef2/users/389ae44e6cfd4f5fbe56063e5b0ae787/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:54:50.276530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31081e-73d2-4347-843f-c2b76390ac8a None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.276930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31081e-73d2-4347-843f-c2b76390ac8a None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.277290 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31081e-73d2-4347-843f-c2b76390ac8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.297940 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-fa31081e-73d2-4347-843f-c2b76390ac8a None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:50.299147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-fa31081e-73d2-4347-843f-c2b76390ac8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:50.478458 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1356/2729] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:54:50.488200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:50.489907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:50.550835 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f2a78-3b27-45ca-b22e-ab5ea7502095 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:50.554632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:50.555258 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:50.555543 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:50.555818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:50.568661 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:50.570118 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:50.579456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1374/2730] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:54.481499 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.482032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.482127 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.514787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] MFA Rules not processed for user `389ae44e6cfd4f5fbe56063e5b0ae787`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:54.516050 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:54.591320 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1357/2731] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:54.594073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.614079 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ecf7ca-4b25-4f3a-8982-ddc038bf2f7c service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.619206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.619996 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.620259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.620531 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.621156 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:54.623099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:54.654185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1375/2732] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:54:54 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:54.658261 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.671854 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0976e3-f527-42f1-91aa-d5c559f47824 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.737693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.738436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.738772 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.739109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.739798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:54.741732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:54.759553 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:54.769402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1358/2733] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:54:54 2022] POST /identity/v3/users => generated 296 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:54:54.770640 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.781775 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7a6a80-7caa-4fc5-a205-d2c37813c163 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.786651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.787267 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.787580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.787925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.788487 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:54.790148 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:54.801990 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1376/2734] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:54.806196 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.817085 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742f989-9c21-4486-b711-fc8ec53f2f66 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.822080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.822759 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.823043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.823318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] PATH_INFO: `/v3/projects/c18706d34bef4581a9508165d117f7fe/users/354c0f4dcd9746d980602861eae51ac1/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.839783 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18706d34bef4581a9508165d117f7fe, user_id=354c0f4dcd9746d980602861eae51ac1, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:54.842425 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:54.859046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1359/2735] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:54 2022] PUT /identity/v3/projects/c18706d34bef4581a9508165d117f7fe/users/354c0f4dcd9746d980602861eae51ac1/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:54.863687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.874135 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682ff26-1fe1-4b8a-ae80-f609d30178f3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.937182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.937182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.937182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.937681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.938266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:54.939629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:54.953249 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1377/2736] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:54:54.957043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.971895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:54.973437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012e9ce-89c8-46df-bc81-c197c309f2e6 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.975357 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:54.984757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af49d903-7c2d-4aee-a0d1-2ece06480ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.985443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49d903-7c2d-4aee-a0d1-2ece06480ddd admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.985745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49d903-7c2d-4aee-a0d1-2ece06480ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:54.986018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af49d903-7c2d-4aee-a0d1-2ece06480ddd admin admin] PATH_INFO: `/v3/projects/c18706d34bef4581a9508165d117f7fe/users/354c0f4dcd9746d980602861eae51ac1/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:54.988527 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49d903-7c2d-4aee-a0d1-2ece06480ddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c18706d34bef4581a9508165d117f7fe, user_id=354c0f4dcd9746d980602861eae51ac1, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:54.991198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af49d903-7c2d-4aee-a0d1-2ece06480ddd admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:54.993031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b2912-1875-4e58-a32c-6b976e93f8bc admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:54.998603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:54.999464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:54.999895 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:55.000318 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] PATH_INFO: `/v3/users/4737f56d9d874539b1468870b0903a5e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:55.002337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4737f56d9d874539b1468870b0903a5e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:55.002515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1360/2737] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:54:54 2022] PUT /identity/v3/projects/c18706d34bef4581a9508165d117f7fe/users/354c0f4dcd9746d980602861eae51ac1/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:55.010320 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:55.072570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4737f56d9d874539b1468870b0903a5e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.074222 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4737f56d9d874539b1468870b0903a5e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.079593 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:54:55.100006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4737f56d9d874539b1468870b0903a5e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.102754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4737f56d9d874539b1468870b0903a5e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.127236 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4737f56d9d874539b1468870b0903a5e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.147806 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1378/2738] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:54:54 2022] DELETE /identity/v3/users/4737f56d9d874539b1468870b0903a5e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:55.152224 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:55.152453 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:55.152723 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:55.195855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:55.197349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:55.298051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1361/2739] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:55.307652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:55.309371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:55.381072 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247d5399-ffdf-4503-9b06-4403da6c1d87 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:55.384870 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f2aa7-e76b-4120-b47f-2e4c73871d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:55.385625 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f2aa7-e76b-4120-b47f-2e4c73871d2f service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:55.385863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f2aa7-e76b-4120-b47f-2e4c73871d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:55.386260 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f2aa7-e76b-4120-b47f-2e4c73871d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:55.398008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f2aa7-e76b-4120-b47f-2e4c73871d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:55.399541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f2aa7-e76b-4120-b47f-2e4c73871d2f service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:55.411089 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1379/2740] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:54:55.734467 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:55.735970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:55.753860 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09f22d-ba41-4bd4-a0e2-e4d7ea7bb971 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:54:55.758924 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:54:55.759582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:55.759883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:55.760183 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] PATH_INFO: `/v3/projects/02260c0eb59e42a581e20db9d0f12f21` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:55.761416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02260c0eb59e42a581e20db9d0f12f21)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:54:55.773808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:54:55.846809 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02260c0eb59e42a581e20db9d0f12f21', 'request_id': 'req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.856605 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02260c0eb59e42a581e20db9d0f12f21', 'request_id': 'req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.864733 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02260c0eb59e42a581e20db9d0f12f21', 'request_id': 'req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.884712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] The token cache is being invalidate because project 02260c0eb59e42a581e20db9d0f12f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:54:55.885125 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.886111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:54:55.888173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1362/2741] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:54:55 2022] DELETE /identity/v3/projects/02260c0eb59e42a581e20db9d0f12f21 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:54:59.819832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec2cb-c613-48e3-a334-48e1b8425adf None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:54:59.820773 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec2cb-c613-48e3-a334-48e1b8425adf None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:54:59.821054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aec2cb-c613-48e3-a334-48e1b8425adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:54:59.855199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-c5aec2cb-c613-48e3-a334-48e1b8425adf None None] MFA Rules not processed for user `354c0f4dcd9746d980602861eae51ac1`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:54:59.856447 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-c5aec2cb-c613-48e3-a334-48e1b8425adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:54:59.934406 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1380/2742] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:54:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:54:59.948032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:54:59.949616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:00.002438 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:00.062911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b9d50-b2b1-40ab-89f8-8a2a1c54c178 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:00.069483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:00.070157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:00.070429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:00.070887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:00.072684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:00.136008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:00.137681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7171264-59d2-469e-8a23-5205700268f4 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:00.147031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1363/2743] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:54:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:55:05.574505 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:05.575102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:05.575102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:05.595533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] MFA Rules not processed for user `962fc6a6c7c04d6f93109c420c5aa6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:55:05.596925 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:05.657121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1381/2744] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:55:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:55:10.143211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=48 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:15.650268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:DELETE=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:16.723357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:16.723906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:16.723906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:16.743838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] MFA Rules not processed for user `389ae44e6cfd4f5fbe56063e5b0ae787`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:55:16.745074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:16.752581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1364/2745] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:55:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:55:26.737646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:32.014390 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:55:32.015805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:32.071504 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daf0a2-ce72-4aeb-a7d8-d73879230440 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:32.075033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:32.075629 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:32.075905 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:32.076173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:32.084843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:32.086355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:32.096007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1382/2746] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:55:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:55:42.091419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=26 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:55:53.367724 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:55:53.369134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:53.425093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b8f8a-6cf0-46ed-8226-3fe1c12347f3 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:55:53.428662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:55:53.429257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:55:53.429538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:55:53.429849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:55:53.431562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:55:53.444787 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:55:53.446277 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:55:53.457431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1365/2747] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:55:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:02.728224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:02.729688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:02.742579 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67908c-34d5-4e7f-8bf3-a205d9b3b017 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:02.746243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:02.746881 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:02.747164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:02.747441 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:02.749225 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:02.763016 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:02.764434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:02.775492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1383/2748] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:03.452371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=26 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:12.770665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:24.881274 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:24.882808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:24.949215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b09574a-cdc1-4bbb-ac77-d7146d0e9db6 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:24.953751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:24.954377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:24.954686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:24.954959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] PATH_INFO: `/v3/users/962fc6a6c7c04d6f93109c420c5aa6cd` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:24.956062 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=962fc6a6c7c04d6f93109c420c5aa6cd)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:24.958088 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.015972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '962fc6a6c7c04d6f93109c420c5aa6cd'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.016655 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '962fc6a6c7c04d6f93109c420c5aa6cd'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.024070 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '962fc6a6c7c04d6f93109c420c5aa6cd'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.024810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '962fc6a6c7c04d6f93109c420c5aa6cd'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.032868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-2dcc05f4-bfa2-4a82-b7d2-7be9c7960f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '962fc6a6c7c04d6f93109c420c5aa6cd'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.050148 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1366/2749] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:56:24 2022] DELETE /identity/v3/users/962fc6a6c7c04d6f93109c420c5aa6cd => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:25.220748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.235347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acba3a37-9f41-41d4-a84a-053d919b5596 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.295456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.296093 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.296378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.296651 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] PATH_INFO: `/v3/projects/b93f8c24b94841168ae2212ae15ecde9` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.297837 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b93f8c24b94841168ae2212ae15ecde9)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.299736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.345493 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b93f8c24b94841168ae2212ae15ecde9', 'request_id': 'req-638e3475-decc-4498-8516-5eb21dcf88ba'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.352194 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b93f8c24b94841168ae2212ae15ecde9', 'request_id': 'req-638e3475-decc-4498-8516-5eb21dcf88ba'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.360216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b93f8c24b94841168ae2212ae15ecde9', 'request_id': 'req-638e3475-decc-4498-8516-5eb21dcf88ba'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.371676 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] The token cache is being invalidate because project b93f8c24b94841168ae2212ae15ecde9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:56:25.371961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.372588 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-638e3475-decc-4498-8516-5eb21dcf88ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:25.374069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1384/2750] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:56:25 2022] DELETE /identity/v3/projects/b93f8c24b94841168ae2212ae15ecde9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:25.379574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.379871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.380150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.399607 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:25.400822 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:25.459916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1367/2751] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:25.467692 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.467977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.468278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.488173 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:25.489391 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:25.497213 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1385/2752] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:25.501487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.511091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935d3ce-e105-46ff-9b5f-dc9a9d29295c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.515512 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.516102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.516380 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.516648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.517293 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:25.517619 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.519241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.531089 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1368/2753] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:25.535409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.545791 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c44f730-53b0-48f8-9342-d4326537aa8f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.550589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.551218 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.551510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.551781 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.552416 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.554362 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.596230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1386/2754] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:56:25 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:56:25.600894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.611115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a33cad-a2c4-44d3-8171-16304ef0950b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.614768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:25.670834 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.671751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.671894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.672072 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.672738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.674856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.695719 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1369/2755] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:56:25 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:25.700000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.710715 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e73a4-2841-4acb-88c7-1452ab704d87 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.715353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.715948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.716223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.716491 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.717120 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.718725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.729142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1387/2756] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:56:25.733408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.743615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6738b9-57ee-43c8-bb2c-b53cb9704921 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.748220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c97c8e2-bb75-4747-9a26-4d83ace53f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.748815 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97c8e2-bb75-4747-9a26-4d83ace53f65 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.749100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97c8e2-bb75-4747-9a26-4d83ace53f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.749378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c97c8e2-bb75-4747-9a26-4d83ace53f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.750060 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97c8e2-bb75-4747-9a26-4d83ace53f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.751659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c97c8e2-bb75-4747-9a26-4d83ace53f65 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.762282 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1370/2757] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:56:25.766408 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.777188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6dcc1-04f2-4fc5-a0c1-34803f6ee74a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:25.781883 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:25.782582 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.782891 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.783161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] PATH_INFO: `/v3/projects/5d32487a82854812803e77fb483323b9/users/e49c8c61fc974f209a3a05f67e8e5790/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.798750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d32487a82854812803e77fb483323b9, user_id=e49c8c61fc974f209a3a05f67e8e5790, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:25.801378 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:25.814392 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1388/2758] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:56:25 2022] PUT /identity/v3/projects/5d32487a82854812803e77fb483323b9/users/e49c8c61fc974f209a3a05f67e8e5790/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:25.825042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:25.825353 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:25.825665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:25.853798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] MFA Rules not processed for user `e49c8c61fc974f209a3a05f67e8e5790`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:25.854943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:25.918147 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1371/2759] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:25.929608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:25.931400 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:25.951750 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:26.005554 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc3fcbe-b305-469b-9579-ad09cd86b807 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:26.011080 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c420a-d5d3-4415-8821-2c17cc363e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:26.011681 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c420a-d5d3-4415-8821-2c17cc363e39 service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:26.011959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c420a-d5d3-4415-8821-2c17cc363e39 service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:26.012229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c420a-d5d3-4415-8821-2c17cc363e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:26.021031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c420a-d5d3-4415-8821-2c17cc363e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:26.022688 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c420a-d5d3-4415-8821-2c17cc363e39 service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:26.032466 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1389/2760] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:25 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:33.298712 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:33.360075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dcd86-0546-489c-a410-7d67ff889bee None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:33.364543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:33.365165 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:33.365442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:33.365742 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] PATH_INFO: `/v3/users/e49c8c61fc974f209a3a05f67e8e5790` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:33.366855 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e49c8c61fc974f209a3a05f67e8e5790)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:33.368763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:33.384796 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:33.416474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e49c8c61fc974f209a3a05f67e8e5790'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.417120 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e49c8c61fc974f209a3a05f67e8e5790'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.424048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e49c8c61fc974f209a3a05f67e8e5790'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.424696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e49c8c61fc974f209a3a05f67e8e5790'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.432731 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e49c8c61fc974f209a3a05f67e8e5790'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.450162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1372/2761] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:56:33 2022] DELETE /identity/v3/users/e49c8c61fc974f209a3a05f67e8e5790 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:33.454807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:33.455265 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:33.455375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:33.474597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:33.475729 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:33.537253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1390/2762] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:33.547501 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:33.548901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:33.605889 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92934a-3a32-4afe-aab9-e82318222f4b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:33.609769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194cbfa4-a2e3-4de8-88db-9dbaae734213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:33.610370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cbfa4-a2e3-4de8-88db-9dbaae734213 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:33.610644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cbfa4-a2e3-4de8-88db-9dbaae734213 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:33.610916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194cbfa4-a2e3-4de8-88db-9dbaae734213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:33.619830 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cbfa4-a2e3-4de8-88db-9dbaae734213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:33.621243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194cbfa4-a2e3-4de8-88db-9dbaae734213 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:33.631099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1373/2763] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:33.903494 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:33.905069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:33.917869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99470a8d-c068-4f18-bc63-919eacade13e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:33.922911 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:33.923630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:33.923976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:33.924310 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] PATH_INFO: `/v3/projects/5d32487a82854812803e77fb483323b9` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:33.925631 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d32487a82854812803e77fb483323b9)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:33.928102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:33.980185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d32487a82854812803e77fb483323b9', 'request_id': 'req-3cd1e8a4-fcb7-4451-a289-8686537beaaa'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.986112 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d32487a82854812803e77fb483323b9', 'request_id': 'req-3cd1e8a4-fcb7-4451-a289-8686537beaaa'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:33.993962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d32487a82854812803e77fb483323b9', 'request_id': 'req-3cd1e8a4-fcb7-4451-a289-8686537beaaa'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:34.005687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] The token cache is being invalidate because project 5d32487a82854812803e77fb483323b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:56:34.005980 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:34.006626 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-3cd1e8a4-fcb7-4451-a289-8686537beaaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:56:34.008109 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1391/2764] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:56:33 2022] DELETE /identity/v3/projects/5d32487a82854812803e77fb483323b9 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:34.012939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.013220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.013511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.032251 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:34.033435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:34.091461 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1374/2765] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:34.101142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.101429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.101811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.121914 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:34.123117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:34.131014 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1392/2766] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:34.135405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.145157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492689ec-ba4c-49c6-89f3-9080c2ac5277 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.149817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.150418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.150692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.150958 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.151604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:56:34.151918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.153525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.165907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1375/2767] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:56:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:56:34.170259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.179976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01375635-2c08-4099-bb56-019a9c8aaab8 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.184649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.185241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.185514 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.185818 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.186432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.188309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.209224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:34.217709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1393/2768] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:34.221404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.231751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf18de-cefd-4145-95df-6b57e000a556 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.252337 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:34.290489 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.291100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.291378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.291650 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.292267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.295172 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.315807 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1376/2769] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:34.320031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.330477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b202d62-d17e-4767-81cb-8c3a72ac22ad admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.335652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.336251 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.336533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.336808 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.337480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.339105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.349710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1394/2770] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:56:34.353670 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.364558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63698f87-6896-47f8-910d-bd604497791c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.369280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a7342c-2bec-447b-b073-1ce5557c47e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.369939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7342c-2bec-447b-b073-1ce5557c47e5 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.370221 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7342c-2bec-447b-b073-1ce5557c47e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.370494 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a7342c-2bec-447b-b073-1ce5557c47e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.371155 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7342c-2bec-447b-b073-1ce5557c47e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.372825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a7342c-2bec-447b-b073-1ce5557c47e5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.385432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1377/2771] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:56:34.389987 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.400469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ca579-a983-44bf-82a2-ad173ed76e1c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.405280 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.406008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.406290 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.406565 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] PATH_INFO: `/v3/projects/321fd89bc182430ca2b17cd3c7e7ffaf/users/b4214314f5fb4795a9b0161f2bfbc196/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.425459 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] RBAC: Authorizing `identity:create_grant(project_id=321fd89bc182430ca2b17cd3c7e7ffaf, user_id=b4214314f5fb4795a9b0161f2bfbc196, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.428908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.443488 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1395/2772] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:56:34 2022] PUT /identity/v3/projects/321fd89bc182430ca2b17cd3c7e7ffaf/users/b4214314f5fb4795a9b0161f2bfbc196/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:34.454713 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.454994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.455267 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.487863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] MFA Rules not processed for user `b4214314f5fb4795a9b0161f2bfbc196`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:34.489477 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:34.555761 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1378/2773] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:34.560435 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.623693 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978152eb-2059-4e3e-bb21-4c79bed4366a admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.628377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.628983 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.629373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.629538 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.630201 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.632095 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.669977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1396/2774] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:56:34.674191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.685075 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe94aa-66ed-4ec3-8b9a-19405e9e2204 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.740616 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.741122 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.741407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.741709 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.742366 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.744265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.765219 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1379/2775] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:56:34.769264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.779939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfecd3-a090-4057-809a-c8dad65b0efb admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.784672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.785298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.786200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.786200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.787090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.788713 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.799828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1397/2776] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:56:34.804146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.813976 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f61f81c-e34f-4feb-aae4-1e92f6a8ae71 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.818656 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.819305 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.819593 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.819864 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] PATH_INFO: `/v3/projects/63cc7c3d08bb49709acddec28d4cd07d/users/6bf72fdfec9943bda57ce55f99af6e8e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.837162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cc7c3d08bb49709acddec28d4cd07d, user_id=6bf72fdfec9943bda57ce55f99af6e8e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.839835 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.848714 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:34.854994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1380/2777] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:56:34 2022] PUT /identity/v3/projects/63cc7c3d08bb49709acddec28d4cd07d/users/6bf72fdfec9943bda57ce55f99af6e8e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:34.860055 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.872329 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920eeb-aa18-42c0-a5d5-359b2d463a64 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.931672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851ff21-5a13-4d9e-ade3-60c3d9597051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.932278 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851ff21-5a13-4d9e-ade3-60c3d9597051 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.932558 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851ff21-5a13-4d9e-ade3-60c3d9597051 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.932824 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851ff21-5a13-4d9e-ade3-60c3d9597051 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.933470 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851ff21-5a13-4d9e-ade3-60c3d9597051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.935068 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0851ff21-5a13-4d9e-ade3-60c3d9597051 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.945456 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1398/2778] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:56:34.949701 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:34.960749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f312cc-1417-4ef2-86af-441a8778fc40 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:34.966253 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:34.966880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.967158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.967430 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] PATH_INFO: `/v3/projects/63cc7c3d08bb49709acddec28d4cd07d/users/6bf72fdfec9943bda57ce55f99af6e8e/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:34.969362 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cc7c3d08bb49709acddec28d4cd07d, user_id=6bf72fdfec9943bda57ce55f99af6e8e, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:34.972052 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:34.982312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1381/2779] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:56:34 2022] PUT /identity/v3/projects/63cc7c3d08bb49709acddec28d4cd07d/users/6bf72fdfec9943bda57ce55f99af6e8e/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:56:34.992830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:34.993045 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:34.993315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.022029 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] MFA Rules not processed for user `6bf72fdfec9943bda57ce55f99af6e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:56:35.023241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:35.042757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:35.090659 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1399/2780] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:56:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:56:35.101825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:35.103520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:56:35.173173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831409f-f965-413c-9d80-32295546dbe3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:35.176996 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:35.177636 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:35.177923 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:35.178195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.187302 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:35.188757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:35.198755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1382/2781] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:35.275578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:56:35.285382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc28a4e-be22-4886-b790-2eee6334c9b2 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:56:35.289027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:56:35.289665 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:56:35.289948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:56:35.290220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:56:35.346048 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:56:35.347469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:56:35.356224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1400/2782] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:56:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:56:45.194509 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:56:45.352212 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=47 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:04.753237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:04.754884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:04.816775 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d6f8c7-dc9d-4b44-a6cf-9b3fc56cf038 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:04.821207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:04.821848 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:04.822134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:04.822405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:04.824189 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:04.834870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:04.837740 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465fa8d-171a-43dc-a1e5-0504d20e6c7b service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:04.846452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1383/2783] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:13.577888 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:13.641155 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9450b43f-ea6c-49de-b3f7-4eec51605c26 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:13.645779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:13.646419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:13.646701 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:13.646974 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] PATH_INFO: `/v3/users/b4214314f5fb4795a9b0161f2bfbc196` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:13.648091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4214314f5fb4795a9b0161f2bfbc196)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:13.650586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:13.697235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4214314f5fb4795a9b0161f2bfbc196'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:13.697913 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4214314f5fb4795a9b0161f2bfbc196'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:13.705312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4214314f5fb4795a9b0161f2bfbc196'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:13.705994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4214314f5fb4795a9b0161f2bfbc196'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:13.718331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4214314f5fb4795a9b0161f2bfbc196'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:13.731224 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1401/2784] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:13 2022] DELETE /identity/v3/users/b4214314f5fb4795a9b0161f2bfbc196 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:13.736013 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:13.736514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:13.736514 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:13.755812 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:13.757016 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:13.814250 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1384/2785] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:13.824898 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:13.826331 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:13.884738 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d459688-3eb6-43ac-b5d1-8d4d979294b9 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:13.888414 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:13.889019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:13.889298 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:13.889586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:13.899636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:13.902244 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:13.914608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1402/2786] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:14.183098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:14.184581 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:14.197828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7005f-fa43-49d5-9cb9-37ba7b154b4c None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:14.202334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:14.202963 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:14.203234 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:14.203503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] PATH_INFO: `/v3/projects/321fd89bc182430ca2b17cd3c7e7ffaf` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:14.204631 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=321fd89bc182430ca2b17cd3c7e7ffaf)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:14.206679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:14.253002 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321fd89bc182430ca2b17cd3c7e7ffaf', 'request_id': 'req-06732825-434d-479b-bf46-f709585207a7'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.259101 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321fd89bc182430ca2b17cd3c7e7ffaf', 'request_id': 'req-06732825-434d-479b-bf46-f709585207a7'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.265225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321fd89bc182430ca2b17cd3c7e7ffaf', 'request_id': 'req-06732825-434d-479b-bf46-f709585207a7'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.279528 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] The token cache is being invalidate because project 321fd89bc182430ca2b17cd3c7e7ffaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:14.279814 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.280418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.281915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1385/2787] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:14 2022] DELETE /identity/v3/projects/321fd89bc182430ca2b17cd3c7e7ffaf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:14.285841 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:14.295399 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90864e-c2ea-48f8-b21d-0a0a88e673f9 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:14.313461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:14.353264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:14.353917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:14.354196 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:14.354467 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] PATH_INFO: `/v3/users/6bf72fdfec9943bda57ce55f99af6e8e` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:14.355578 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf72fdfec9943bda57ce55f99af6e8e)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:14.357489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:14.403526 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf72fdfec9943bda57ce55f99af6e8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.404177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf72fdfec9943bda57ce55f99af6e8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.411465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf72fdfec9943bda57ce55f99af6e8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.412253 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf72fdfec9943bda57ce55f99af6e8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.424584 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-c44caa10-9656-4c3b-9947-7f7c1602f680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf72fdfec9943bda57ce55f99af6e8e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.436992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1403/2788] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:14 2022] DELETE /identity/v3/users/6bf72fdfec9943bda57ce55f99af6e8e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:14.706579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:14.708020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:14.721445 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06732825-434d-479b-bf46-f709585207a7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:14.747842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:14.780816 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:14.781377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:14.781694 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:14.781971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] PATH_INFO: `/v3/projects/63cc7c3d08bb49709acddec28d4cd07d` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:14.783084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63cc7c3d08bb49709acddec28d4cd07d)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:14.784999 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:14.831849 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63cc7c3d08bb49709acddec28d4cd07d', 'request_id': 'req-5467cd9f-1133-4e97-b880-18bd0bf20af4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.837920 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63cc7c3d08bb49709acddec28d4cd07d', 'request_id': 'req-5467cd9f-1133-4e97-b880-18bd0bf20af4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.844069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63cc7c3d08bb49709acddec28d4cd07d', 'request_id': 'req-5467cd9f-1133-4e97-b880-18bd0bf20af4'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.856157 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] The token cache is being invalidate because project 63cc7c3d08bb49709acddec28d4cd07d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:14.856443 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.857067 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-5467cd9f-1133-4e97-b880-18bd0bf20af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:14.858856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1386/2789] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:14 2022] DELETE /identity/v3/projects/63cc7c3d08bb49709acddec28d4cd07d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:14.863716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:14.864174 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:14.864285 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:14.883566 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:14.884757 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:14.944656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1404/2790] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:14.952406 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:14.952684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:14.952952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:14.973074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:14.974319 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:14.983094 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1387/2791] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:14.987279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:14.997021 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193714ef-657e-40a1-a959-69143d1ca93f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.001616 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.002221 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.002501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.002774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.003427 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:15.003724 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.005330 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.016903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1405/2792] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:15.021220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.030930 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0a53f-3b02-4079-b8db-60909c38e6a5 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.035468 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.036042 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.036317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.036586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.037198 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.039166 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.067523 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1388/2793] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:57:15.071638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.081761 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd92a56-6708-4aa2-8e6c-cf90639dc6ed admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.109369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:15.142010 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.142622 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.142903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.143178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.143886 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.145892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.167289 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1406/2794] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:57:15.171479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.181931 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423462dc-13f7-4561-897b-b11cd857a172 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.186384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.186971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.187245 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.187511 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.188137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.189773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.199436 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1389/2795] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:15.203210 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.213515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4020601-7865-476b-a90d-db64b8815695 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.218150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f1e2c-5bb7-40d5-8787-108199ec4c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.218746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f1e2c-5bb7-40d5-8787-108199ec4c87 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.219027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f1e2c-5bb7-40d5-8787-108199ec4c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.219309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f1e2c-5bb7-40d5-8787-108199ec4c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.219942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f1e2c-5bb7-40d5-8787-108199ec4c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.221527 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f1e2c-5bb7-40d5-8787-108199ec4c87 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.233099 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1407/2796] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:15.237111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.247503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14630b75-83cf-4db9-b2c4-038b6ba04490 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.252109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.252758 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.253037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.253310 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] PATH_INFO: `/v3/projects/b6d7113694ad49e891bee7c99c8208b9/users/d33482caac0045e8bea4b703b8deb230/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.268696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d7113694ad49e891bee7c99c8208b9, user_id=d33482caac0045e8bea4b703b8deb230, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.271408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.285381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1390/2797] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:15 2022] PUT /identity/v3/projects/b6d7113694ad49e891bee7c99c8208b9/users/d33482caac0045e8bea4b703b8deb230/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:15.296413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.296703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.296981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.324921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] MFA Rules not processed for user `d33482caac0045e8bea4b703b8deb230`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:15.326154 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:15.390563 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1408/2798] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:15.395112 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.445917 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:15.457887 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ecf53-9717-4a61-9ad2-6d84bf1a9d37 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.462799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.463402 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.463678 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.463948 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.464562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.466485 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.494105 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1391/2799] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:15.498256 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.509022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3b735-a805-4527-8846-88204af9c916 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.564902 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.565429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.565746 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.566033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.566650 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.568562 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.590846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1409/2800] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 05:57:15.595031 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.605798 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445bfc3-a804-48cf-bd5c-6597b686fe63 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.610717 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.611317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.611598 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.611877 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.612519 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.614563 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.625225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1392/2801] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:57:15.629477 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.639696 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20569334-07cd-46e3-bc08-60155e657d6b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.644274 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.644924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.645202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.645469 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] PATH_INFO: `/v3/projects/83f8b58c355748c0af4da30b9514bdb5/users/182395ea2fda4e2caab281c7a17234d8/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.660199 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f8b58c355748c0af4da30b9514bdb5, user_id=182395ea2fda4e2caab281c7a17234d8, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.662859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.676670 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1410/2802] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:15 2022] PUT /identity/v3/projects/83f8b58c355748c0af4da30b9514bdb5/users/182395ea2fda4e2caab281c7a17234d8/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:15.681391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.691357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd420036-d779-4d2a-b94e-1019b9a57764 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.746937 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f9a3d-c6a9-40bd-8286-46bcb4d65d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.747571 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9a3d-c6a9-40bd-8286-46bcb4d65d35 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.747853 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9a3d-c6a9-40bd-8286-46bcb4d65d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.748404 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f9a3d-c6a9-40bd-8286-46bcb4d65d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.748763 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f9a3d-c6a9-40bd-8286-46bcb4d65d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.750390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f9a3d-c6a9-40bd-8286-46bcb4d65d35 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.760134 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1393/2803] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:15.764266 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.769970 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:15.777229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec174a5-cd91-4d81-a60d-2179265fb0d3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:15.782200 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:15.782845 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.783119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.783385 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] PATH_INFO: `/v3/projects/83f8b58c355748c0af4da30b9514bdb5/users/182395ea2fda4e2caab281c7a17234d8/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.785262 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f8b58c355748c0af4da30b9514bdb5, user_id=182395ea2fda4e2caab281c7a17234d8, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:15.787927 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:15.798371 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1411/2804] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:15 2022] PUT /identity/v3/projects/83f8b58c355748c0af4da30b9514bdb5/users/182395ea2fda4e2caab281c7a17234d8/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:15.862646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:15.862960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:15.863237 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:15.890738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] MFA Rules not processed for user `182395ea2fda4e2caab281c7a17234d8`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:15.891967 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:15.958693 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1394/2805] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:15.971420 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:15.974091 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:16.031648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f0651-3464-47dd-a850-edce23a0ae0b admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.035315 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.035918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.036202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.036484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.045474 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.046948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.056486 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1412/2806] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:15 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:16.363171 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.372875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b73f9-07cd-4a75-8c6e-7e91e896f0b4 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.377479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182395ea2fda4e2caab281c7a17234d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f8b58c355748c0af4da30b9514bdb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.378043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.378334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.378613 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.379263 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:16.379568 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.381160 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.393188 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1395/2807] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:16.397311 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.406586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f64925-fc87-4bd1-8c44-bbb102473607 service cinder] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.411904 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182395ea2fda4e2caab281c7a17234d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f8b58c355748c0af4da30b9514bdb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.412497 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.412776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.413075 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.413740 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.415703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.442867 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1413/2808] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:16 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:16.447762 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.457743 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75217bd1-4774-4e77-9dd3-98cad4796dfa tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.489121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:16.515285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182395ea2fda4e2caab281c7a17234d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f8b58c355748c0af4da30b9514bdb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.515913 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.516200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.516475 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.517106 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:16.517408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.519033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.530873 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1396/2809] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:16.567894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.579397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45209e59-dc8f-42df-9281-9665827b059e tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.584102 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '182395ea2fda4e2caab281c7a17234d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f8b58c355748c0af4da30b9514bdb5', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.584734 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.585017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.585292 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] PATH_INFO: `/v3/projects/521f855482094ec1a57c805bff84dd4f` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.586743 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorizing `identity:delete_project(project_id=521f855482094ec1a57c805bff84dd4f)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.588780 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:16.637917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '521f855482094ec1a57c805bff84dd4f', 'request_id': 'req-007e136f-6f0a-4937-b637-b97af9c63cb0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:16.643433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '521f855482094ec1a57c805bff84dd4f', 'request_id': 'req-007e136f-6f0a-4937-b637-b97af9c63cb0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:16.650932 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '521f855482094ec1a57c805bff84dd4f', 'request_id': 'req-007e136f-6f0a-4937-b637-b97af9c63cb0'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:16.663387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] The token cache is being invalidate because project 521f855482094ec1a57c805bff84dd4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:16.663716 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:16.664404 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-007e136f-6f0a-4937-b637-b97af9c63cb0 tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:16.666058 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1414/2810] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:16 2022] DELETE /identity/v3/projects/521f855482094ec1a57c805bff84dd4f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:16.857644 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:16.859635 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:16.924301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e411-9c3c-49f9-863f-319ef9bacacf tempest-VolumeQuotaClassesTest-576405761 tempest-VolumeQuotaClassesTest-576405761-project-admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:16.979415 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:16.980046 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:16.980327 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:16.980603 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] PATH_INFO: `/v3/users/d33482caac0045e8bea4b703b8deb230` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:16.981773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33482caac0045e8bea4b703b8deb230)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:16.983954 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.029824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33482caac0045e8bea4b703b8deb230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.030474 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33482caac0045e8bea4b703b8deb230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.037229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33482caac0045e8bea4b703b8deb230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.037870 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33482caac0045e8bea4b703b8deb230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.045757 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33482caac0045e8bea4b703b8deb230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.057355 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.065479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1397/2811] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:16 2022] DELETE /identity/v3/users/d33482caac0045e8bea4b703b8deb230 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:17.070202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.070708 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.070812 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.089857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:17.091059 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:17.139428 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:17.153438 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1415/2812] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:17.163084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.165093 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:17.220865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d699d-c056-4c80-965a-a6c72abc31de admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.224391 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.224986 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.225262 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.225537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.235352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.236776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.246301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1398/2813] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:17.504805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.506337 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:17.521877 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88c780-8a02-4dea-bb73-8e59f080342d None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.530424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.530979 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.531259 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.531577 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] PATH_INFO: `/v3/projects/b6d7113694ad49e891bee7c99c8208b9` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.532655 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d7113694ad49e891bee7c99c8208b9)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.534647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.580007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d7113694ad49e891bee7c99c8208b9', 'request_id': 'req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.585345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d7113694ad49e891bee7c99c8208b9', 'request_id': 'req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.593506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d7113694ad49e891bee7c99c8208b9', 'request_id': 'req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.604422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] The token cache is being invalidate because project b6d7113694ad49e891bee7c99c8208b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:17.604704 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.605319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.606797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1416/2814] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:17 2022] DELETE /identity/v3/projects/b6d7113694ad49e891bee7c99c8208b9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:17.610988 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:17.620416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f4fc2-6ce0-4cde-8041-9e6f6872dbd7 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:17.677328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:17.678326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:17.678606 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:17.678888 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] PATH_INFO: `/v3/users/182395ea2fda4e2caab281c7a17234d8` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:17.680043 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=182395ea2fda4e2caab281c7a17234d8)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:17.682003 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:17.728074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '182395ea2fda4e2caab281c7a17234d8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.728727 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '182395ea2fda4e2caab281c7a17234d8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.735842 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '182395ea2fda4e2caab281c7a17234d8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.736496 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '182395ea2fda4e2caab281c7a17234d8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.743906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-942358a3-e5d5-4eed-92b3-7874d90c9961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '182395ea2fda4e2caab281c7a17234d8'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:17.761326 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1399/2815] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:17 2022] DELETE /identity/v3/users/182395ea2fda4e2caab281c7a17234d8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:18.026117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.027570 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:18.042855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42b1fc-5f50-47d5-b945-485d2ae9d51c admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.100698 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.101380 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.101639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.101912 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] PATH_INFO: `/v3/projects/83f8b58c355748c0af4da30b9514bdb5` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.103034 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f8b58c355748c0af4da30b9514bdb5)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.104918 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.153073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f8b58c355748c0af4da30b9514bdb5', 'request_id': 'req-f5a13487-41dc-48c5-93d7-c5973077df4e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:18.158537 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f8b58c355748c0af4da30b9514bdb5', 'request_id': 'req-f5a13487-41dc-48c5-93d7-c5973077df4e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:18.165890 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f8b58c355748c0af4da30b9514bdb5', 'request_id': 'req-f5a13487-41dc-48c5-93d7-c5973077df4e'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:18.176811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] The token cache is being invalidate because project 83f8b58c355748c0af4da30b9514bdb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:18.177105 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:18.177830 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-f5a13487-41dc-48c5-93d7-c5973077df4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:18.179345 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1417/2816] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:18 2022] DELETE /identity/v3/projects/83f8b58c355748c0af4da30b9514bdb5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:18.184907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.185429 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.185543 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.205567 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:18.206803 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:18.218903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.271276 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1400/2817] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:18.278760 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.279041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.279336 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.294220 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.299807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:18.301635 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:18.310992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1418/2818] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:18.315074 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.324885 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6a44d-6deb-40b3-b980-c999c9feaa84 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.329488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.330110 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.330390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.330661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.331400 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:18.331708 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.333296 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.344473 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1401/2819] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:18.348831 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.358838 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb71916-3c14-4fc1-ade8-d12a74831ceb None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.363432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.364017 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.364291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.364557 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.365180 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.367092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.394344 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1419/2820] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:18.398586 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.408687 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813764ab-7a9e-4707-955a-8e2e549fbfe9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.464356 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.464952 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.465232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.465504 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.466159 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.468063 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.490144 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1402/2821] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:18.494816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.504805 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e32faa-f30c-40c6-88d3-4d79feacdc38 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.509601 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.510312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.510591 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.510860 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.511492 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.513092 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.523087 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1420/2822] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:18.527116 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.537291 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f118778-723c-4bd7-aa98-7897519690c7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.542426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344afa4-7590-4b13-b913-023801ecb7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.543020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344afa4-7590-4b13-b913-023801ecb7b2 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.543299 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344afa4-7590-4b13-b913-023801ecb7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.543573 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344afa4-7590-4b13-b913-023801ecb7b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.544229 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344afa4-7590-4b13-b913-023801ecb7b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.545833 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344afa4-7590-4b13-b913-023801ecb7b2 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.558396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1403/2823] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:18.562432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.572364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d3f1b-4b50-4e89-b9e7-0cac5531452e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.576948 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.577614 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.577894 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.578161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] PATH_INFO: `/v3/projects/93b884edfd9142c088d16401ecef9fb3/users/faaa327114c5452c8461c7278ab21263/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.592636 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93b884edfd9142c088d16401ecef9fb3, user_id=faaa327114c5452c8461c7278ab21263, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.595313 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.610161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1421/2824] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:18 2022] PUT /identity/v3/projects/93b884edfd9142c088d16401ecef9fb3/users/faaa327114c5452c8461c7278ab21263/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:18.621285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.621592 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.621878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.650416 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] MFA Rules not processed for user `faaa327114c5452c8461c7278ab21263`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:18.651615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:18.714564 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1404/2825] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:18.719115 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.783185 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff822f0-f659-4813-95dc-efeba14d1296 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.788241 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.788855 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.789764 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.790041 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.790682 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.792583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.821397 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1422/2826] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:18.823239 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.834428 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e393e-fb6f-4d6a-a837-df1fedee911b None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.843533 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:18.891407 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.892008 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.892288 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.892558 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.893173 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.895126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.917328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1405/2827] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:18 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:18.921323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.931541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c3f4a-4543-49d2-97e3-664f620f4e40 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.936182 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.936783 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.937064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.937335 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.938006 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.939630 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.950375 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1423/2828] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:18.954344 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.964737 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c46c43-146c-4013-8adf-c4a65e81d89a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:18.969311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a2fa5-d288-46b7-b355-617ecb5e85b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:18.969927 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a2fa5-d288-46b7-b355-617ecb5e85b5 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:18.970207 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a2fa5-d288-46b7-b355-617ecb5e85b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:18.970480 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a2fa5-d288-46b7-b355-617ecb5e85b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:18.971146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a2fa5-d288-46b7-b355-617ecb5e85b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:18.972725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a2fa5-d288-46b7-b355-617ecb5e85b5 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:18.986417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1406/2829] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:18.988737 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:18.998691 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86beb184-46de-4a19-8b3f-07a9d68a6828 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.003275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.003931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.004208 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.004481 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] PATH_INFO: `/v3/projects/f8397ba1ade0463d807d6bde2c7f9063/users/b306485e772d46f5b98b007b950ff230/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.019355 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8397ba1ade0463d807d6bde2c7f9063, user_id=b306485e772d46f5b98b007b950ff230, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.022036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.036121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1424/2830] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:18 2022] PUT /identity/v3/projects/f8397ba1ade0463d807d6bde2c7f9063/users/b306485e772d46f5b98b007b950ff230/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:19.046283 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.046574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.046852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.074505 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] MFA Rules not processed for user `b306485e772d46f5b98b007b950ff230`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:19.075793 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:19.139136 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1407/2831] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:19.143603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.194223 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.205168 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9afee5-5fb5-4917-a573-360669ac589e admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.209992 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.210590 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.210892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.211767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.212386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.214312 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.240553 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1425/2832] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:19 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:19.244825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.255451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92713f4-19ab-44f7-850b-dc7d05d8ae18 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.313832 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.314370 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.314658 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.314956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.315582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.317722 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.340292 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1408/2833] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:19 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:19.344373 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.354675 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aa6132-d314-477f-99a2-540d9bfd042f admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.359515 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.360113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.360396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.360672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.361316 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.363033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.374117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1426/2834] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:19.378114 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.388790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42be32-d1e0-4ebe-8b02-b2655af2d1b9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.393424 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.394097 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.394378 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.394646 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] PATH_INFO: `/v3/projects/7158528c1f924819801de300757d9bc4/users/47346450b99f4526a0b800a79da91956/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.403019 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.410270 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7158528c1f924819801de300757d9bc4, user_id=47346450b99f4526a0b800a79da91956, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.413091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.425301 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1409/2835] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:19 2022] PUT /identity/v3/projects/7158528c1f924819801de300757d9bc4/users/47346450b99f4526a0b800a79da91956/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:19.429522 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.439941 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197e7f9-9df1-42f8-b884-1fa3ce1ddd80 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.496968 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dfb902-cf19-448e-a0c5-cea175e9eeca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.497589 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfb902-cf19-448e-a0c5-cea175e9eeca admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.497879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfb902-cf19-448e-a0c5-cea175e9eeca admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.498150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dfb902-cf19-448e-a0c5-cea175e9eeca admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.498779 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfb902-cf19-448e-a0c5-cea175e9eeca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.500364 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dfb902-cf19-448e-a0c5-cea175e9eeca admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.510686 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1427/2836] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:19.514876 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.525211 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb474fd-e2e5-4c2d-a6bc-751bbe5e462a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.529841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.530486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.530766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.531038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] PATH_INFO: `/v3/projects/7158528c1f924819801de300757d9bc4/users/47346450b99f4526a0b800a79da91956/roles/5ccd18fa93fc4e9f9428ee63d11deba5` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.532936 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7158528c1f924819801de300757d9bc4, user_id=47346450b99f4526a0b800a79da91956, role_id=5ccd18fa93fc4e9f9428ee63d11deba5)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.535595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.545503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1410/2837] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:19 2022] PUT /identity/v3/projects/7158528c1f924819801de300757d9bc4/users/47346450b99f4526a0b800a79da91956/roles/5ccd18fa93fc4e9f9428ee63d11deba5 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:19.555977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.556254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.556523 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.583871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] MFA Rules not processed for user `47346450b99f4526a0b800a79da91956`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:19.585076 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:19.663907 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1428/2838] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:19.675317 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.677300 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:19.734525 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2fb90-ee0e-4403-9b0b-babc6bae9d14 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.738145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.738738 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.739017 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.739287 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.748788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.750243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.759403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1411/2839] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:19.800015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:19.801674 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:19.814230 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd4808-0a23-4eda-a0c9-e013947f51ab None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:19.817908 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34d2f79-1d7e-43bc-b328-95f88ae60885 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:19.818500 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d2f79-1d7e-43bc-b328-95f88ae60885 service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:19.818782 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d2f79-1d7e-43bc-b328-95f88ae60885 service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:19.819054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34d2f79-1d7e-43bc-b328-95f88ae60885 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:19.865899 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:19.878419 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d2f79-1d7e-43bc-b328-95f88ae60885 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:19.879849 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34d2f79-1d7e-43bc-b328-95f88ae60885 service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:19.888689 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1429/2840] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:22.760820 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:22.820905 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc0907-b50f-4ed2-8364-4ad2ff10213a service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:22.825437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:22.826084 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.826357 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.826625 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] PATH_INFO: `/v3/users/faaa327114c5452c8461c7278ab21263` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.827732 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faaa327114c5452c8461c7278ab21263)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:22.829668 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:22.875968 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faaa327114c5452c8461c7278ab21263'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.876618 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faaa327114c5452c8461c7278ab21263'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.883352 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faaa327114c5452c8461c7278ab21263'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.884012 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faaa327114c5452c8461c7278ab21263'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.891423 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faaa327114c5452c8461c7278ab21263'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:22.902135 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:22.909841 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1412/2841] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:22 2022] DELETE /identity/v3/users/faaa327114c5452c8461c7278ab21263 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 05:57:22.914319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:22.914826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:22.914939 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:22.934842 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:22.935938 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:22.994025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1430/2842] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:57:23.003623 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.005032 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:23.064092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb736c5f-dac7-4c2f-9e8a-4ab50257e3e8 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.067766 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.068375 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.068657 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.068932 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.077883 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.079309 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.089028 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1413/2843] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:23.355777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.357254 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:23.371309 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce02414-d623-40dd-9626-bb125bd7116b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.376356 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.376994 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.377294 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.377585 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] PATH_INFO: `/v3/projects/93b884edfd9142c088d16401ecef9fb3` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.378945 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93b884edfd9142c088d16401ecef9fb3)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.382036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.429025 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93b884edfd9142c088d16401ecef9fb3', 'request_id': 'req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.435709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93b884edfd9142c088d16401ecef9fb3', 'request_id': 'req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.443150 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93b884edfd9142c088d16401ecef9fb3', 'request_id': 'req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.455919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] The token cache is being invalidate because project 93b884edfd9142c088d16401ecef9fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:23.456216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.456859 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.458448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1431/2844] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:23 2022] DELETE /identity/v3/projects/93b884edfd9142c088d16401ecef9fb3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:23.462435 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.471939 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065334e6-5424-4b96-b027-f5ac49ac6b61 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.526141 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.526777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.527051 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.527321 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] PATH_INFO: `/v3/users/b306485e772d46f5b98b007b950ff230` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.528418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b306485e772d46f5b98b007b950ff230)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.530359 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.576788 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b306485e772d46f5b98b007b950ff230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.577427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b306485e772d46f5b98b007b950ff230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.584128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b306485e772d46f5b98b007b950ff230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.584745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b306485e772d46f5b98b007b950ff230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.592419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b306485e772d46f5b98b007b950ff230'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:23.610427 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1414/2845] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:23 2022] DELETE /identity/v3/users/b306485e772d46f5b98b007b950ff230 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:23.878533 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:23.880047 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:23.893921 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad4853-7d74-4592-bb0d-c4a012f0f8a5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:23.951159 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:23.951844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:23.952053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:23.952325 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] PATH_INFO: `/v3/projects/f8397ba1ade0463d807d6bde2c7f9063` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:23.953432 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8397ba1ade0463d807d6bde2c7f9063)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:23.955353 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:23.966634 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.003284 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8397ba1ade0463d807d6bde2c7f9063', 'request_id': 'req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.008597 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8397ba1ade0463d807d6bde2c7f9063', 'request_id': 'req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.017608 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8397ba1ade0463d807d6bde2c7f9063', 'request_id': 'req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.028790 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] The token cache is being invalidate because project f8397ba1ade0463d807d6bde2c7f9063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:24.029081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.029755 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.031238 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1432/2846] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:23 2022] DELETE /identity/v3/projects/f8397ba1ade0463d807d6bde2c7f9063 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:24.035246 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.036756 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.049265 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33bfd82-b578-4c45-81e1-174c67be7dfd admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.062033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.108257 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.108897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.109178 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.109452 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] PATH_INFO: `/v3/users/47346450b99f4526a0b800a79da91956` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.110604 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47346450b99f4526a0b800a79da91956)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.112537 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.163280 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47346450b99f4526a0b800a79da91956'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.163981 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47346450b99f4526a0b800a79da91956'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.173784 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47346450b99f4526a0b800a79da91956'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.174521 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47346450b99f4526a0b800a79da91956'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.182865 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-0e1e0332-14b1-435b-922c-6d23487bb610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47346450b99f4526a0b800a79da91956'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.200777 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1415/2847] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:24 2022] DELETE /identity/v3/users/47346450b99f4526a0b800a79da91956 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:24.473343 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.474959 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.492106 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5cd49b-ef75-4fb0-b61f-9264f4dc2c0d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.549909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.550461 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.550744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.551019 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] PATH_INFO: `/v3/projects/7158528c1f924819801de300757d9bc4` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.552140 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7158528c1f924819801de300757d9bc4)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.554086 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.599774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7158528c1f924819801de300757d9bc4', 'request_id': 'req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.605952 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7158528c1f924819801de300757d9bc4', 'request_id': 'req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.613250 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7158528c1f924819801de300757d9bc4', 'request_id': 'req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.624953 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] The token cache is being invalidate because project 7158528c1f924819801de300757d9bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:24.625243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.625887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-afc722f6-c12d-4cbc-8a77-b5eaf9f05a7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:24.627383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1433/2848] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:24 2022] DELETE /identity/v3/projects/7158528c1f924819801de300757d9bc4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:24.633520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.633831 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.634133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.653856 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:24.655038 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.714648 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1416/2849] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:57:24.724449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.724735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.725013 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.745243 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:24.746480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:24.754374 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1434/2850] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:24.758741 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.768431 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4757e75-a225-44d9-950c-b0d292fb53ef None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.773243 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.773868 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.774145 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.774417 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.775836 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88145) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:24.776137 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.777745 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.789294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1417/2851] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:24.794229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.803662 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d0e269-6b12-4ea1-9128-2dee0c97d38b None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.808206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.808784 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.809060 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.809327 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] PATH_INFO: `/v3/projects` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.810005 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.811887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.840843 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1435/2852] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:24 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:24.845030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.855442 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a4868-d056-45cc-83fb-c333f5a620b9 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.876191 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.915092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.916269 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.916554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.916825 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] PATH_INFO: `/v3/users` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.917446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.919486 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.942289 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1418/2853] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:24 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:24.946940 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.952370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:24.958288 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393383fc-aa8b-4467-9ba0-6a53c95e1b1d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.965202 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:24.965997 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:24.966445 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:24.966832 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:24.967726 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:24.969326 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:24.979561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1436/2854] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 05:57:24.983863 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:24.995215 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9427826-a9ce-44b6-9781-f8a9b6c94569 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:24.999960 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc99d7-8331-4512-8fb1-8562d7a61054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.000557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc99d7-8331-4512-8fb1-8562d7a61054 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.000838 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc99d7-8331-4512-8fb1-8562d7a61054 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.001111 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc99d7-8331-4512-8fb1-8562d7a61054 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.001773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc99d7-8331-4512-8fb1-8562d7a61054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.003396 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc99d7-8331-4512-8fb1-8562d7a61054 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:25.014554 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1419/2855] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:25.018735 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.029954 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ccb2dc-0669-4a74-9b3c-ddd1cbe1cf21 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.034712 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.035370 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.035652 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.035924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] PATH_INFO: `/v3/projects/14aedc1de2ff4c0596f391c30f6aff25/users/c82a7df3648c42cba8f159f90ebed247/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.050365 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=14aedc1de2ff4c0596f391c30f6aff25, user_id=c82a7df3648c42cba8f159f90ebed247, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.053064 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:25.067161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1437/2856] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:25 2022] PUT /identity/v3/projects/14aedc1de2ff4c0596f391c30f6aff25/users/c82a7df3648c42cba8f159f90ebed247/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:25.078840 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.079318 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.079418 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.107254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] MFA Rules not processed for user `c82a7df3648c42cba8f159f90ebed247`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:25.108725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:25.172205 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1420/2857] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:25.183142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:25.184846 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:25.244117 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a074bbaa-7534-4e35-86d8-27d636d7fd4d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:25.248291 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223ef775-a87f-4926-ba45-5050c97c38cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:25.248900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef775-a87f-4926-ba45-5050c97c38cc service cinder] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:25.249179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef775-a87f-4926-ba45-5050c97c38cc service cinder] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:25.249448 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223ef775-a87f-4926-ba45-5050c97c38cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:25.258297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef775-a87f-4926-ba45-5050c97c38cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:25.260261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223ef775-a87f-4926-ba45-5050c97c38cc service cinder] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:25.269451 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1438/2858] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:28.269377 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.333649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6c249-06dc-4a84-b332-fc4ba294b1bb None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.338367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.338998 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.339279 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.339552 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] PATH_INFO: `/v3/users/c82a7df3648c42cba8f159f90ebed247` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.340720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82a7df3648c42cba8f159f90ebed247)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.342773 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.385335 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:28.396379 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82a7df3648c42cba8f159f90ebed247'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:28.397087 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82a7df3648c42cba8f159f90ebed247'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:28.404146 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82a7df3648c42cba8f159f90ebed247'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:28.404810 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82a7df3648c42cba8f159f90ebed247'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:28.412176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82a7df3648c42cba8f159f90ebed247'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:28.430176 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1421/2859] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:57:28 2022] DELETE /identity/v3/users/c82a7df3648c42cba8f159f90ebed247 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:28.435413 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.435900 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.436001 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.455510 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:28.456736 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:28.518261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1439/2860] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:28.528295 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.529824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:28.589454 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66225e40-0395-4dc0-b4d3-411bd86934ee admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.593054 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.595057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.595057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.595057 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.604538 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.606232 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:28.617483 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1422/2861] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:28.929113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.932901 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:28.933884 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:28.934614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:28.952613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27fc0a-ad2b-4fdc-a394-152fa3406c9e None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:28.958503 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:28.959319 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:28.959677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:28.960027 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] PATH_INFO: `/v3/projects/14aedc1de2ff4c0596f391c30f6aff25` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:28.961334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14aedc1de2ff4c0596f391c30f6aff25)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:28.963468 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.045768 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14aedc1de2ff4c0596f391c30f6aff25', 'request_id': 'req-317c1440-d02e-4c93-ba59-56651bfadb16'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:29.053286 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14aedc1de2ff4c0596f391c30f6aff25', 'request_id': 'req-317c1440-d02e-4c93-ba59-56651bfadb16'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:29.060048 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4e516d-cd4f-4ea0-b1cd-c9eac0f65347 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.063826 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14aedc1de2ff4c0596f391c30f6aff25', 'request_id': 'req-317c1440-d02e-4c93-ba59-56651bfadb16'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:29.066053 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:29.091270 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] The token cache is being invalidate because project 14aedc1de2ff4c0596f391c30f6aff25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:57:29.091580 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:29.092395 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-317c1440-d02e-4c93-ba59-56651bfadb16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:57:29.097282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1440/2862] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:57:28 2022] DELETE /identity/v3/projects/14aedc1de2ff4c0596f391c30f6aff25 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:29.100387 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.100811 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.101190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.126639 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:29.127778 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:29.127928 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:29.156109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.156346 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.156614 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.156894 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.159142 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:29.174439 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.176192 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5ce824-647e-4dda-bb55-95b7ca6a0624 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.189371 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1423/2863] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:57:29.219709 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1441/2864] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:29.220989 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.221268 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.221540 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.258336 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:29.260329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:29.273696 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1424/2865] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:57:29.276749 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.287264 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51698e4-a012-4655-8aab-ed8c09a71b5f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.292966 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.293612 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.293903 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.294179 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.294857 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:57:29.295178 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.296852 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.311031 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1442/2866] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:57:29.315630 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.325363 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084c5a42-e912-440e-af4e-e1e7eb1814f2 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.330200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.330875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.331092 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.331368 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.332158 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.333982 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.366837 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1425/2867] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:57:29 2022] POST /identity/v3/projects => generated 363 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:29.371753 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.382649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8a15c2-ed49-43e1-a50d-03c4c28cefb3 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.443754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.444431 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.444649 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.444924 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.445748 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.448642 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.474607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1443/2868] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:57:29 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:57:29.475891 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.487264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e83279-4091-4e6a-983e-3081d463330d admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.491915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.492596 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.492938 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.493909 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.494617 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.496338 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.509979 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1426/2869] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:29.511393 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.521422 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc3ec81-0edf-4dfd-ad04-64fb9bb80ce2 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.526097 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4067b6-1c4d-411a-a8b6-ba476eea66dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.526725 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4067b6-1c4d-411a-a8b6-ba476eea66dd admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.527007 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4067b6-1c4d-411a-a8b6-ba476eea66dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.527282 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4067b6-1c4d-411a-a8b6-ba476eea66dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.527920 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4067b6-1c4d-411a-a8b6-ba476eea66dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.529511 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4067b6-1c4d-411a-a8b6-ba476eea66dd admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.540865 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1444/2870] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:57:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:57:29.545264 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.556706 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41289f9-cf9b-4a87-a314-df98a8b46841 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.561479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.562163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.562508 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.562790 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] PATH_INFO: `/v3/projects/834626e291ed4f75be99ed663000d167/users/1361bc1cb4bf4e83b47d57ba90b3f059/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.577662 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=834626e291ed4f75be99ed663000d167, user_id=1361bc1cb4bf4e83b47d57ba90b3f059, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.580316 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.593530 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1427/2871] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:57:29 2022] PUT /identity/v3/projects/834626e291ed4f75be99ed663000d167/users/1361bc1cb4bf4e83b47d57ba90b3f059/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:57:29.604261 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896b17-8bf8-48a5-b885-7c525d33ad3f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.604550 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896b17-8bf8-48a5-b885-7c525d33ad3f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.604828 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2896b17-8bf8-48a5-b885-7c525d33ad3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.634015 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-b2896b17-8bf8-48a5-b885-7c525d33ad3f None None] MFA Rules not processed for user `1361bc1cb4bf4e83b47d57ba90b3f059`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:29.635164 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-b2896b17-8bf8-48a5-b885-7c525d33ad3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:29.688464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:29.718062 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1445/2872] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:29.728943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:29.730869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:29.816869 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6d7c1-074c-4bff-981b-398f645e3bbb admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:29.820769 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:29.821520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:29.821910 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:29.822256 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:29.832328 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:29.833878 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:29.847169 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1428/2873] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:30.911915 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:30.912368 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:30.912480 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:30.932524 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] MFA Rules not processed for user `389ae44e6cfd4f5fbe56063e5b0ae787`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:57:30.933807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:30.986815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1446/2874] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:57:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:57:31.309911 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:31.311367 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:31.376432 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95481b8-7a4c-401f-b379-2522b79fa1e8 service cinder] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:31.380045 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:31.380649 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:31.380929 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:31.381202 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:31.382972 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:31.393642 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:31.395076 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:31.404000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1429/2875] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:32.903033 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:32.904517 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:32.977679 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b0fdc-b7bd-42d9-81e1-46902d15a859 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:32.982767 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:32.983638 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:32.983916 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:32.984396 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:32.986175 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:32.999205 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:33.000797 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:33.012113 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1447/2876] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:57:41.400104 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:43.007548 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=62 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:57:52.005871 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:57:52.007312 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:52.087181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e91fb7-ce3b-44d4-98b5-979ad263dee7 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:57:52.093725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:57:52.094349 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:57:52.094632 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:57:52.094906 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:57:52.096666 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:57:52.109588 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:57:52.111182 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943accae-b18a-41bd-b058-8dcbf1450e60 service nova] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:57:52.121645 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1430/2877] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:57:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 05:58:02.117194 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=34 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:04.965125 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.034507 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a4757-4543-4633-bbaa-22311b3791eb service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.039011 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.039672 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.039946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.040214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] PATH_INFO: `/v3/users/1361bc1cb4bf4e83b47d57ba90b3f059` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.041333 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1361bc1cb4bf4e83b47d57ba90b3f059)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.043279 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.093475 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1361bc1cb4bf4e83b47d57ba90b3f059'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.094766 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1361bc1cb4bf4e83b47d57ba90b3f059'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.102231 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1361bc1cb4bf4e83b47d57ba90b3f059'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.102917 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1361bc1cb4bf4e83b47d57ba90b3f059'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.116317 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1361bc1cb4bf4e83b47d57ba90b3f059'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.129603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1448/2878] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:04 2022] DELETE /identity/v3/users/1361bc1cb4bf4e83b47d57ba90b3f059 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:05.134200 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.134718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.134718 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.154595 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:05.155922 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.224992 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1431/2879] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:05.237409 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.239163 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.304000 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0a54-13d4-4811-af3d-19f4410f375d admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.307770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f620cf-2b3b-4e9c-9d88-7a97bde0fa50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.308377 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f620cf-2b3b-4e9c-9d88-7a97bde0fa50 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.308656 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f620cf-2b3b-4e9c-9d88-7a97bde0fa50 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.308977 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f620cf-2b3b-4e9c-9d88-7a97bde0fa50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.318976 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f620cf-2b3b-4e9c-9d88-7a97bde0fa50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.320433 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f620cf-2b3b-4e9c-9d88-7a97bde0fa50 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.333816 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1449/2880] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:05.619351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.621115 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.639686 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11034ff7-7699-4cda-8da0-70237768e37f None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.653638 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.654401 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.654755 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.655099 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] PATH_INFO: `/v3/projects/834626e291ed4f75be99ed663000d167` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.657018 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=834626e291ed4f75be99ed663000d167)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.659334 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.708943 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '834626e291ed4f75be99ed663000d167', 'request_id': 'req-76766ec1-9313-418e-8357-f7a902b2a863'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.714410 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '834626e291ed4f75be99ed663000d167', 'request_id': 'req-76766ec1-9313-418e-8357-f7a902b2a863'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.720661 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '834626e291ed4f75be99ed663000d167', 'request_id': 'req-76766ec1-9313-418e-8357-f7a902b2a863'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.735098 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] The token cache is being invalidate because project 834626e291ed4f75be99ed663000d167 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:05.735388 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.736009 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-76766ec1-9313-418e-8357-f7a902b2a863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:05.737513 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1432/2881] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:05 2022] DELETE /identity/v3/projects/834626e291ed4f75be99ed663000d167 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:05.742506 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.742796 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.743073 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.762929 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:05.764153 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.780878 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:05.824541 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1450/2882] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:05.833875 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.834162 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.834488 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.853679 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:05.854903 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:05.863971 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1433/2883] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 05:58:05.868235 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.878909 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179f988-4dbe-4244-9d74-588e8f99bbef None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.883710 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.884323 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.884603 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.884875 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.885528 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88146) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 05:58:05.885869 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.887463 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.899347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1451/2884] 10.222.0.73 () {58 vars in 1153 bytes} [Wed Aug 24 05:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 05:58:05.903748 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.913150 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18d61e-1aa3-479b-8c9b-d4b9cd851338 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:05.917808 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:05.918403 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:05.918684 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:05.918956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] PATH_INFO: `/v3/projects` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:05.919579 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:05.921479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:05.949884 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1434/2885] 10.222.0.73 () {60 vars in 1153 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:05.954275 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:05.964367 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d363b2f4-95c9-46af-ae93-240795852ec5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.021718 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.022363 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.022586 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.022879 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] PATH_INFO: `/v3/users` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.023495 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.025437 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.047981 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1452/2886] 10.222.0.73 () {60 vars in 1144 bytes} [Wed Aug 24 05:58:05 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 05:58:06.052342 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.062594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa964e7-e017-4973-b2bd-60d27ee7a62b admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.067163 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.067754 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.068037 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.068311 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] PATH_INFO: `/v3/roles` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.068956 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.070583 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.080254 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1435/2887] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:06.084161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.094972 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca686b-77c3-4504-9bad-a17a4a90edfb admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.099607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bc739-3fff-4e87-b25f-2cefebbc215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.100206 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc739-3fff-4e87-b25f-2cefebbc215f admin admin] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.100489 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc739-3fff-4e87-b25f-2cefebbc215f admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.100765 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bc739-3fff-4e87-b25f-2cefebbc215f admin admin] PATH_INFO: `/v3/roles` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.101402 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc739-3fff-4e87-b25f-2cefebbc215f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.103022 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bc739-3fff-4e87-b25f-2cefebbc215f admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.114567 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1453/2888] 10.222.0.73 () {58 vars in 1122 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 05:58:06.118653 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.128852 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a0863-80c1-4a31-bd93-6a960e3ec28c admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.133591 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.134419 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.135000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.135000 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] PATH_INFO: `/v3/projects/1fac603499764ff8bd058c6c785b0c70/users/a522ce33905c41999ba169d98f343944/roles/e6fe411570314ffdb43c23bed35b1f50` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.149437 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fac603499764ff8bd058c6c785b0c70, user_id=a522ce33905c41999ba169d98f343944, role_id=e6fe411570314ffdb43c23bed35b1f50)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.152225 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.165640 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1436/2889] 10.222.0.73 () {60 vars in 1483 bytes} [Wed Aug 24 05:58:06 2022] PUT /identity/v3/projects/1fac603499764ff8bd058c6c785b0c70/users/a522ce33905c41999ba169d98f343944/roles/e6fe411570314ffdb43c23bed35b1f50 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:06.176561 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.176840 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.177137 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.205787 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] MFA Rules not processed for user `a522ce33905c41999ba169d98f343944`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:06.207347 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:06.273162 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1454/2890] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:58:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:06.284306 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:06.286030 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:06.308079 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:06.344451 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecff0cf-38f3-4a1a-bb5b-144336518fa3 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:06.348152 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ea571-f73b-4dd7-b1ae-41f8a4fffa42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c7f12a5dc6446eaa94a96dbb44ccdaf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:06.348751 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ea571-f73b-4dd7-b1ae-41f8a4fffa42 service cinder] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:06.349033 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ea571-f73b-4dd7-b1ae-41f8a4fffa42 service cinder] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:06.349307 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ea571-f73b-4dd7-b1ae-41f8a4fffa42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:06.357446 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ea571-f73b-4dd7-b1ae-41f8a4fffa42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:06.359180 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ea571-f73b-4dd7-b1ae-41f8a4fffa42 service cinder] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:06.368383 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1437/2891] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:16.270306 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:16.364390 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=17 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:23.753611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.814188 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc5ce6-56c8-4a36-b7be-bf9c3e7c9e0f None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:23.819324 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:23.819962 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.820214 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.820501 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] PATH_INFO: `/v3/users/a522ce33905c41999ba169d98f343944` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.821632 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a522ce33905c41999ba169d98f343944)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:23.823535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:23.868853 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a522ce33905c41999ba169d98f343944'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:23.869499 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a522ce33905c41999ba169d98f343944'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:23.876774 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a522ce33905c41999ba169d98f343944'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:23.878297 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a522ce33905c41999ba169d98f343944'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:23.890277 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a522ce33905c41999ba169d98f343944'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:23.902607 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1455/2892] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 05:58:23 2022] DELETE /identity/v3/users/a522ce33905c41999ba169d98f343944 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:23.907080 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] REQUEST_METHOD: `POST` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:23.907557 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:23.907659 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:23.926590 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.auth.core [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] MFA Rules not processed for user `a3d3f4f0c68b4b679ab49519fa363f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88145) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:58:23.927799 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:23.985824 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1438/2893] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:58:23.995382 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:23.996807 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:24.052573 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c678dda-ad16-4808-a3f5-f8726fed70cb admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:24.056216 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6ed49-e38e-496a-8c24-361f7735632c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:24.056815 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6ed49-e38e-496a-8c24-361f7735632c service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.057100 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6ed49-e38e-496a-8c24-361f7735632c service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.057358 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6ed49-e38e-496a-8c24-361f7735632c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.066690 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6ed49-e38e-496a-8c24-361f7735632c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:24.068104 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6ed49-e38e-496a-8c24-361f7735632c service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:24.077677 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1456/2894] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:58:24.341983 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:58:24.343515 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:58:24.355953 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051f042-24b0-4cf4-aed7-c55411b778b0 None None] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:58:24.360826 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:58:24.361994 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:58:24.362273 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:58:24.362577 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] PATH_INFO: `/v3/projects/1fac603499764ff8bd058c6c785b0c70` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:58:24.363704 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fac603499764ff8bd058c6c785b0c70)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:58:24.365725 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:58:24.412329 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fac603499764ff8bd058c6c785b0c70', 'request_id': 'req-75c49faf-d6e5-4db1-8ef6-359fd736344f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:24.417683 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fac603499764ff8bd058c6c785b0c70', 'request_id': 'req-75c49faf-d6e5-4db1-8ef6-359fd736344f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:24.424384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fac603499764ff8bd058c6c785b0c70', 'request_id': 'req-75c49faf-d6e5-4db1-8ef6-359fd736344f'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:24.436285 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] The token cache is being invalidate because project 1fac603499764ff8bd058c6c785b0c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88145) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 05:58:24.436574 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:24.437195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 05:58:24.438744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1439/2895] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 05:58:24 2022] DELETE /identity/v3/projects/1fac603499764ff8bd058c6c785b0c70 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 05:58:34.073887 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:58:34.433100 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:43.105777 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] REQUEST_METHOD: `POST` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:43.106334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:43.106334 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:43.124892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.auth.core [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] MFA Rules not processed for user `389ae44e6cfd4f5fbe56063e5b0ae787`. Rule list: `[]` (Enabled: `True`). {{(pid=88146) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 05:59:43.126142 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:43.179307 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1457/2896] 10.222.0.73 () {58 vars in 958 bytes} [Wed Aug 24 05:59:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 05:59:44.379916 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 05:59:44.381339 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:44.450589 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49faf-d6e5-4db1-8ef6-359fd736344f admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 05:59:44.455749 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 05:59:44.456381 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 05:59:44.456663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 05:59:44.456934 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 05:59:44.458692 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 05:59:44.470001 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 05:59:44.471582 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 05:59:44.481964 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1440/2897] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 05:59:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 05:59:53.172619 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=25 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 05:59:54.477681 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=32 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:00:00.051613 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:00:00.053054 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:00.139081 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d5863-67a2-417e-a3a0-186d20012816 None None] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:00:00.145819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4e926d4a438e42ecaeab44f711e0b3c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:00:00.146484 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:00:00.146770 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:00:00.147040 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:00:00.148844 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:00:00.162776 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:00:00.164386 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:00:00.175958 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1458/2898] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:00:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:00:10.170930 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=34 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:29.174973 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:29.176384 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:29.232699 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961dbf06-665b-43b0-9bac-22723abd73ee service placement] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:29.236786 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:29.237386 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:29.237695 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:29.237970 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:29.239663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:29.250720 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:29.252133 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:29.262817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1441/2899] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:30.262449 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:30.263871 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:30.276819 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4291676-01f7-4b2c-9a99-f4028989547f service nova] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:30.281338 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:30.281975 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:30.282255 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:30.282521 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:30.284229 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:30.298121 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:30.299572 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:30.310802 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1459/2900] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 06:01:31.489314 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:31.498103 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30d006-6dcd-4a52-a715-5ac88de7ec69 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:31.501620 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:31.502206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:31.502479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:31.502744 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:31.504408 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:31.517479 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:31.518915 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:31.530128 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1442/2901] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:40.305663 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=6 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:41.524880 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=32 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:42.325430 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:42.334226 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e270e5-325e-496d-990c-efe5446aa40e service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:42.337982 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '20154db5efc6482bbae149b81fb06cd2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:42.338583 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:42.338863 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:42.339138 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:42.340931 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:42.405090 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:42.406472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:42.416620 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1460/2902] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:45.445102 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:45.454069 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:45.469405 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1092d1-b8f6-4078-83c8-a84f52079a33 service neutron] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:45.474600 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:45.475220 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:45.475503 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:45.475776 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:45.477768 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:45.490907 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:45.492520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:45.505828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1443/2903] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:48.499440 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:48.512610 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c4c8-297a-4077-94d5-76d7cb3ab517 service neutron] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:48.518472 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:48.519177 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] REQUEST_METHOD: `GET` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:48.519465 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:48.519744 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:48.521627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:48.539424 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:48.542190 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:48.556520 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:48.559121 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:48.565771 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1461/2904] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:01:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:01:48.580918 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ad9608-aec5-4869-b01b-f2bc9299f895 service placement] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:48.586209 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:48.586904 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:48.587195 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:48.588181 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] PATH_INFO: `/v3/users/389ae44e6cfd4f5fbe56063e5b0ae787` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:48.589397 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=389ae44e6cfd4f5fbe56063e5b0ae787)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:48.592081 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:48.646020 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389ae44e6cfd4f5fbe56063e5b0ae787'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.646672 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389ae44e6cfd4f5fbe56063e5b0ae787'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.653995 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389ae44e6cfd4f5fbe56063e5b0ae787'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.654663 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389ae44e6cfd4f5fbe56063e5b0ae787'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.662615 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389ae44e6cfd4f5fbe56063e5b0ae787'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.680665 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1444/2905] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 06:01:48 2022] DELETE /identity/v3/users/389ae44e6cfd4f5fbe56063e5b0ae787 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:48.849664 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:48.861429 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0e554-9352-424a-9e64-364c36da26f8 service placement] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:48.919703 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:48.920369 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:48.920648 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:48.920919 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] PATH_INFO: `/v3/projects/76656ddf3c744ba1a32cfaa37e278ef2` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:48.922069 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76656ddf3c744ba1a32cfaa37e278ef2)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:48.923961 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:48.969942 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76656ddf3c744ba1a32cfaa37e278ef2', 'request_id': 'req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.975383 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76656ddf3c744ba1a32cfaa37e278ef2', 'request_id': 'req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.983239 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76656ddf3c744ba1a32cfaa37e278ef2', 'request_id': 'req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.995535 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] The token cache is being invalidate because project 76656ddf3c744ba1a32cfaa37e278ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:01:48.995820 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.996444 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:48.998036 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1462/2906] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 06:01:48 2022] DELETE /identity/v3/projects/76656ddf3c744ba1a32cfaa37e278ef2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:49.002259 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:49.003746 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:49.017897 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab76102-3e3f-43c0-8434-682f287d1d5a admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:49.077867 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:49.078541 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:49.078817 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:49.079091 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] PATH_INFO: `/v3/users/354c0f4dcd9746d980602861eae51ac1` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:49.080241 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=354c0f4dcd9746d980602861eae51ac1)` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:49.082206 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:49.129594 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354c0f4dcd9746d980602861eae51ac1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.130351 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354c0f4dcd9746d980602861eae51ac1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.138702 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354c0f4dcd9746d980602861eae51ac1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.139385 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354c0f4dcd9746d980602861eae51ac1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.143534 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:49.147792 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.notifications [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354c0f4dcd9746d980602861eae51ac1'} {{(pid=88145) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.167426 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1445/2907] 10.222.0.73 () {58 vars in 1224 bytes} [Wed Aug 24 06:01:49 2022] DELETE /identity/v3/users/354c0f4dcd9746d980602861eae51ac1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:49.341687 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Authenticating user token {{(pid=88146) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:01:49.343157 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.fernet_utils [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88146) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:01:49.359008 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc51c6a-b9da-4404-97e0-7ff59e30cab5 admin admin] Validating token access rules against request {{(pid=88146) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:01:49.368699 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:49.420946 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3d3f4f0c68b4b679ab49519fa363f6e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '136c4807ccbf43c49f1d29adcff44403', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88146) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:01:49.421611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:01:49.421892 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:01:49.422161 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] PATH_INFO: `/v3/projects/c18706d34bef4581a9508165d117f7fe` {{(pid=88146) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:01:49.423268 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c18706d34bef4581a9508165d117f7fe)` {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:01:49.425165 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] RBAC: Authorization granted {{(pid=88146) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:01:49.470627 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c18706d34bef4581a9508165d117f7fe', 'request_id': 'req-1c245820-566d-47d1-a844-17789e046d30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.476119 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c18706d34bef4581a9508165d117f7fe', 'request_id': 'req-1c245820-566d-47d1-a844-17789e046d30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.484460 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c18706d34bef4581a9508165d117f7fe', 'request_id': 'req-1c245820-566d-47d1-a844-17789e046d30'} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.495464 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] The token cache is being invalidate because project c18706d34bef4581a9508165d117f7fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88146) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 06:01:49.495754 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.496611 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG keystone.notifications [None req-1c245820-566d-47d1-a844-17789e046d30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88146) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 06:01:49.498434 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: [pid: 88146|app: 0|req: 1463/2908] 10.222.0.73 () {58 vars in 1233 bytes} [Wed Aug 24 06:01:49 2022] DELETE /identity/v3/projects/c18706d34bef4581a9508165d117f7fe => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 06:01:59.166460 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:01:59.492647 dv-cinder-854268-1-iscsi devstack@keystone.service[88146]: DEBUG dbcounter [-] [88146] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88146) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 06:05:08.031874 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Authenticating user token {{(pid=88145) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 06:05:08.033294 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:08.170828 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064133-8341-4c0b-b7aa-c0c8656bc2f7 admin admin] Validating token access rules against request {{(pid=88145) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 06:05:08.229959 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61a7bc1bc86a405a9ef04a3d36502a7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6f233421cc684264987120c578af1f1c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88145) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 06:05:08.230562 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] REQUEST_METHOD: `GET` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 06:05:08.230839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] SCRIPT_NAME: `/identity` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 06:05:08.231109 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88145) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 06:05:08.232912 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.fernet_utils [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88145) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 06:05:08.244839 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 06:05:08.246487 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2a721f-420e-402b-8a80-ff56a835ceb4 service placement] RBAC: Authorization granted {{(pid=88145) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 06:05:08.256440 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: [pid: 88145|app: 0|req: 1446/2909] 10.222.0.73 () {60 vars in 1365 bytes} [Wed Aug 24 06:05:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 06:05:18.252126 dv-cinder-854268-1-iscsi devstack@keystone.service[88145]: DEBUG dbcounter [-] [88145] Writing DB stats keystone:SELECT=89 {{(pid=88145) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}